(req) unsubscribe

[dong.yajun]

list, I'd like to unsubscribe this mail list.

thanks.


-- 
*Ric Dong *

RE: How HBase perform per-column scan?

[Liu, Raymond]

Hmm, I don't mean query bloom filter directly. I mean the storefilescanner will 
query rowcol bloomfilter to see is it need a seek or not. And I guess this will 
be performed on every row without need to specific a row keys?


> ROWCOL bloom says whether for a given row (rowkey) a given column (qualifier)
> is present in an HFile or not.  But for the user he dont know the rowkeys. He
> wants all the rows with column 'x'
> 
> -Anoop-
> 
> 
> From: Liu, Raymond [raymond@intel.com]
> Sent: Monday, March 11, 2013 7:43 AM
> To: user@hbase.apache.org
> Subject: RE: How HBase perform per-column scan?
> 
> Just curious, won't ROWCOL bloom filter works for this case?
> 
> Best Regards,
> Raymond Liu
> 
> >
> > As per the above said, you will need a full table scan on that CF.
> > As Ted said, consider having a look at your schema design.
> >
> > -Anoop-
> >
> >
> > On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu  wrote:
> >
> > > bq. physically column family should be able to perform efficiently
> > > (storage layer
> > >
> > > When you scan a row, data for different column families would be
> > > brought into memory (if you don't utilize HBASE-5416) Take a look at:
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > > 1258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-ta
> > > bp
> > > anel#comment-13541258
> > >
> > > which was based on the settings described in:
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > > 1191&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-ta
> > > bp
> > > anel#comment-13541191
> > >
> > > This boils down to your schema design. If possible, consider
> > > extracting column C into its own column family.
> > >
> > > Cheers
> > >
> > > On Sun, Mar 10, 2013 at 7:14 AM, PG  wrote:
> > >
> > > > Hi, Ted and Anoop, thanks for your notes.
> > > > I am talking about column rather than column family, since
> > > > physically column family should be able to perform efficiently
> > > > (storage layer, CF's are stored separately). But columns of the
> > > > same column family may be
> > > mixed
> > > > physically, and that makes filters column value hard... So I want
> > > > to know if there are any mechanism in HBase worked on this...
> > > > Regards,
> > > > Yun
> > > >
> > > > On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:
> > > >
> > > > > Hi, Yun:
> > > > > Take a look at HBASE-5416 (Improve performance of scans with
> > > > > some kind
> > > of
> > > > > filters) which is in 0.94.5 release.
> > > > >
> > > > > In your case, you can use a filter which specifies column C as
> > > > > the essential family.
> > > > > Here I interpret column C as column family.
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Sat, Mar 9, 2013 at 11:11 AM, yun peng
> > > > > 
> > > wrote:
> > > > >
> > > > >> Hi, All,
> > > > >> I want to find all existing values for a given column in a
> > > > >> HBase, and
> > > > would
> > > > >> that result in a full-table scan in HBase? For example, given a
> > > > >> column
> > > > C,
> > > > >> the table is of very large number of rows, from which few rows
> > > > >> (say
> > > > only 1
> > > > >> row) have non-empty values for column C. Would HBase still ues
> > > > >> a full
> > > > table
> > > > >> scan to find this row? Or HBase has any optimization work for
> > > > >> this
> > > kind
> > > > of
> > > > >> query?
> > > > >> Thanks...
> > > > >> Regards
> > > > >> Yun
> > > > >>
> > > >
> > >

Re: can we use same column name for 2 different column families?

[ramkrishna vasudevan]

As note of caution just don have two empty qualifiers in the same CF but u
can still have empty qualifiers in diff CFs.

Regards
Ram

On Mon, Mar 11, 2013 at 10:20 AM, Anoop Sam John  wrote:

> >can we have column name "dob" under column family "F1" & "F2"?
> Just fine..  Go ahead.. :)
>
> -Anoop-
> 
> From: Ramasubramanian Narayanan [ramasubramanian.naraya...@gmail.com]
> Sent: Sunday, March 10, 2013 11:41 PM
> To: user@hbase.apache.org
> Subject: can we use same column name for 2 different column families?
>
> Hi,
>
> Is it fine to use same column name for 2 different column families?
>
> For example,
>
> In a table "emp",
>
> can we have column name "dob" under column family "F1" & "F2"?
>
> Please let me know the impact of having like this if any...
>
> Note : I don't want to use dob1 or some other field name for the second
> column... use case is like that...
>
> regards,
> Rams
>

RE: can we use same column name for 2 different column families?

[Anoop Sam John]

>can we have column name "dob" under column family "F1" & "F2"?
Just fine..  Go ahead.. :)

-Anoop-

From: Ramasubramanian Narayanan [ramasubramanian.naraya...@gmail.com]
Sent: Sunday, March 10, 2013 11:41 PM
To: user@hbase.apache.org
Subject: can we use same column name for 2 different column families?

Hi,

Is it fine to use same column name for 2 different column families?

For example,

In a table "emp",

can we have column name "dob" under column family "F1" & "F2"?

Please let me know the impact of having like this if any...

Note : I don't want to use dob1 or some other field name for the second
column... use case is like that...

regards,
Rams

RE: How HBase perform per-column scan?

[Anoop Sam John]

ROWCOL bloom says whether for a given row (rowkey) a given column (qualifier) 
is present in an HFile or not.  But for the user he dont know the rowkeys. He 
wants all the rows with column 'x'

-Anoop-


From: Liu, Raymond [raymond@intel.com]
Sent: Monday, March 11, 2013 7:43 AM
To: user@hbase.apache.org
Subject: RE: How HBase perform per-column scan?

Just curious, won't ROWCOL bloom filter works for this case?

Best Regards,
Raymond Liu

>
> As per the above said, you will need a full table scan on that CF.
> As Ted said, consider having a look at your schema design.
>
> -Anoop-
>
>
> On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu  wrote:
>
> > bq. physically column family should be able to perform efficiently
> > (storage layer
> >
> > When you scan a row, data for different column families would be
> > brought into memory (if you don't utilize HBASE-5416) Take a look at:
> >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > 1258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabp
> > anel#comment-13541258
> >
> > which was based on the settings described in:
> >
> >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > 1191&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabp
> > anel#comment-13541191
> >
> > This boils down to your schema design. If possible, consider
> > extracting column C into its own column family.
> >
> > Cheers
> >
> > On Sun, Mar 10, 2013 at 7:14 AM, PG  wrote:
> >
> > > Hi, Ted and Anoop, thanks for your notes.
> > > I am talking about column rather than column family, since
> > > physically column family should be able to perform efficiently
> > > (storage layer, CF's are stored separately). But columns of the same
> > > column family may be
> > mixed
> > > physically, and that makes filters column value hard... So I want to
> > > know if there are any mechanism in HBase worked on this...
> > > Regards,
> > > Yun
> > >
> > > On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:
> > >
> > > > Hi, Yun:
> > > > Take a look at HBASE-5416 (Improve performance of scans with some
> > > > kind
> > of
> > > > filters) which is in 0.94.5 release.
> > > >
> > > > In your case, you can use a filter which specifies column C as the
> > > > essential family.
> > > > Here I interpret column C as column family.
> > > >
> > > > Cheers
> > > >
> > > > On Sat, Mar 9, 2013 at 11:11 AM, yun peng 
> > wrote:
> > > >
> > > >> Hi, All,
> > > >> I want to find all existing values for a given column in a HBase,
> > > >> and
> > > would
> > > >> that result in a full-table scan in HBase? For example, given a
> > > >> column
> > > C,
> > > >> the table is of very large number of rows, from which few rows
> > > >> (say
> > > only 1
> > > >> row) have non-empty values for column C. Would HBase still ues a
> > > >> full
> > > table
> > > >> scan to find this row? Or HBase has any optimization work for
> > > >> this
> > kind
> > > of
> > > >> query?
> > > >> Thanks...
> > > >> Regards
> > > >> Yun
> > > >>
> > >
> >

RE: Welcome our newest Committer Anoop

[rajeshbabu chintaguntla]

Contratulations Anoop! 


From: Anoop Sam John [anoo...@huawei.com]
Sent: Monday, March 11, 2013 9:00 AM
To: user@hbase.apache.org
Subject: RE: Welcome our newest Committer Anoop

Thanks to all.. Hope to work more and more for HBase!

-Anoop-


From: Andrew Purtell [apurt...@apache.org]
Sent: Monday, March 11, 2013 7:33 AM
To: user@hbase.apache.org
Subject: Re: Welcome our newest Committer Anoop

Congratulations Anoop. Welcome!


On Mon, Mar 11, 2013 at 12:42 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
>
> He has contributed features related to Endpoints and CPs.
>
> Welcome Anoop and best wishes for your future work.
>
> Hope to see your continuing efforts to the community.
>
> Regards
> Ram
>



--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

RE: Welcome our newest Committer Anoop

[Uma Maheswara Rao G]

Congrats Anoop. Great work!

Regards,
Uma


From: ramkrishna vasudevan [ramkrishna.s.vasude...@gmail.com]
Sent: Sunday, March 10, 2013 10:12 PM
To: d...@hbase.apache.org; user@hbase.apache.org
Subject: Welcome our newest Committer Anoop

Hi All

Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
great and he has helped lot of users in the mailing list.

He has contributed features related to Endpoints and CPs.

Welcome Anoop and best wishes for your future work.

Hope to see your continuing efforts to the community.

Regards
Ram

RE: Welcome our newest Committer Anoop

[Anoop Sam John]

Thanks to all.. Hope to work more and more for HBase!

-Anoop-


From: Andrew Purtell [apurt...@apache.org]
Sent: Monday, March 11, 2013 7:33 AM
To: user@hbase.apache.org
Subject: Re: Welcome our newest Committer Anoop

Congratulations Anoop. Welcome!


On Mon, Mar 11, 2013 at 12:42 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
>
> He has contributed features related to Endpoints and CPs.
>
> Welcome Anoop and best wishes for your future work.
>
> Hope to see your continuing efforts to the community.
>
> Regards
> Ram
>



--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: RegionServers Crashing every hour in production env

[Andrew Purtell]

Be careful with GC tuning, throwing changes at an application without
analysis of what is going on with the heap is shooting in the dark. One
particular good treatment of the subject is here:
http://java.dzone.com/articles/how-tame-java-gc-pauses

If you have made custom changes to blockcache or memstore configurations,
back them out until you're sure everything else is ok.

Watch carefully for swapping. Set the vm.swappiness sysctl to 0. Monitor
for spikes in page scanning or any swap activity. Nothing brings on
"Juliette" pauses better than a JVM partially swapped out. The Java GC
starts collection by examining the oldest pages, and those are the first
pages the OS swaps out...



On Mon, Mar 11, 2013 at 10:13 AM, Azuryy Yu  wrote:

> Hi Pablo,
> It'a terrible for a long minor GC. I don't think there are swaping from
> your vmstat log.
> but I just suggest you
> 1) add following JVM options:
> -XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:GCTimeRatio=19
> -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=2
> -XX:MaxTenuringThreshold=3 -XX:+UseFastAccessorMethods
>
> 2) -Xmn is two small, your total Mem is 74GB, just make -Xmn2g
> 3) what are you doing during long GC happened? read or write? if reading,
> what the block cache size?
>
>
>
>
> On Mon, Mar 11, 2013 at 6:41 AM, Stack  wrote:
>
> > You could increase your zookeeper session timeout to 5 minutes while you
> > are figuring why these long pauses.
> > http://hbase.apache.org/book.html#zookeeper.session.timeout
> >
> > Above, there is an outage for almost 5 minutes:
> >
> > >> We slept 225100ms instead of 3000ms, this is likely due to a long
> >
> > You have ganglia or tsdb running?  When you see the big pause above, can
> > you see anything going on on the machine?  (swap, iowait, concurrent fat
> > mapreduce job?)
> >
> > St.Ack
> >
> >
> >
> > On Sun, Mar 10, 2013 at 3:29 PM, Pablo Musa  wrote:
> >
> > > Hi Sreepathi,
> > > they say in the book (or the site), we could try it to see if it is
> > really
> > > a timeout error
> > > or there is something more. But it is not recomended for production
> > > environments.
> > >
> > > I could give it a try if five minutes will ensure to us that the
> problem
> > > is the GC or
> > > elsewhere!! Anyway, I think it is hard to beleive a GC is taking 2:30
> > > minutes.
> > >
> > > Abs,
> > > Pablo
> > >
> > >
> > > On 03/10/2013 04:06 PM, Sreepathi wrote:
> > >
> > >> Hi Stack/Ted/Pablo,
> > >>
> > >> Should we increase the hbase.rpc.timeout property to 5 minutes (
> 30
> > ms
> > >> )  ?
> > >>
> > >> Regards,
> > >> - Sreepathi
> > >>
> > >> On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:
> > >>
> > >>  That combo should be fine.
> > 
> > >>> Great!!
> > >>>
> > >>>
> > >>>  If JVM is full GC'ing, the application is stopped.
> >  The below does not look like a full GC but that is a long pause in
> >  system
> >  time, enough to kill your zk session.
> > 
> > >>> Exactly. This pause is really making the zk expire the RS which
> > shutsdown
> > >>> (logs
> > >>> in the end of the email).
> > >>> But the question is: what is causing this pause??!!
> > >>>
> > >>>  You swapping?
> > 
> > >>> I don't think so (stats below).
> > >>>
> > >>>  Hardware is good?
> > 
> > >>> Yes, it is a 16 processor machine with 74GB of RAM and plenty disk
> > space.
> > >>> Below are some metrics I have heard about. Hope it helps.
> > >>>
> > >>>
> > >>> ** I am having some problems with the datanodes[1] which are having
> > >>> trouble to
> > >>> write. I really think the issues are related, but cannot solve any of
> > >>> them
> > >>> :(
> > >>>
> > >>> Thanks again,
> > >>> Pablo
> > >>>
> > >>> [1] http://mail-archives.apache.
> org/mod_mbox/hadoop-hdfs-user/
> > >>>
> 201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr
> > >>> qfv...@mail.gmail.com%3E > >>> mod_mbox/hadoop-hdfs-user/**201303.mbox/%3CCAJzooYfS-F1KS+**
> > >>> jGOPUt15PwFjcCSzigE0APeM9FXaCr**qfv...@mail.gmail.com%3E<
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201303.mbox/%3ccajzooyfs-f1ks+jgoput15pwfjccszige0apem9fxacrqfv...@mail.gmail.com%3E
> > >
> > >>> >
> > >>>
> > >>> top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34,
> 2.55,
> > >>> 1.28
> > >>> Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
> > >>> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
> > >>>   0.0%st
> > >>> Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k
> > buffers
> > >>> Swap: 51609592k total,   128312k used, 51481280k free,  1353400k
> cached
> > >>>
> > >>> ]$ vmstat -w
> > >>> procs ---memory-- ---swap--
> > >>> -io
> > >>> --system-- -cpu---
> > >>>   r  b   swpd   free   buff  cache   si   sobi bo
> > >>> in
> > >>>cs  us sy  id wa st
> > >>>   2  0 128312   324169285838288504356000   202 53
> > >>>  0
> > >>> 0   2  1  96

Re: RegionServers Crashing every hour in production env

[Azuryy Yu]

Pablo,

another, what's your java version?


On Mon, Mar 11, 2013 at 10:13 AM, Azuryy Yu  wrote:

> Hi Pablo,
> It'a terrible for a long minor GC. I don't think there are swaping from
> your vmstat log.
> but I just suggest you
> 1) add following JVM options:
> -XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:GCTimeRatio=19
> -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=2
> -XX:MaxTenuringThreshold=3 -XX:+UseFastAccessorMethods
>
> 2) -Xmn is two small, your total Mem is 74GB, just make -Xmn2g
> 3) what are you doing during long GC happened? read or write? if reading,
> what the block cache size?
>
>
>
>
> On Mon, Mar 11, 2013 at 6:41 AM, Stack  wrote:
>
>> You could increase your zookeeper session timeout to 5 minutes while you
>> are figuring why these long pauses.
>> http://hbase.apache.org/book.html#zookeeper.session.timeout
>>
>> Above, there is an outage for almost 5 minutes:
>>
>> >> We slept 225100ms instead of 3000ms, this is likely due to a long
>>
>> You have ganglia or tsdb running?  When you see the big pause above, can
>> you see anything going on on the machine?  (swap, iowait, concurrent fat
>> mapreduce job?)
>>
>> St.Ack
>>
>>
>>
>> On Sun, Mar 10, 2013 at 3:29 PM, Pablo Musa  wrote:
>>
>> > Hi Sreepathi,
>> > they say in the book (or the site), we could try it to see if it is
>> really
>> > a timeout error
>> > or there is something more. But it is not recomended for production
>> > environments.
>> >
>> > I could give it a try if five minutes will ensure to us that the problem
>> > is the GC or
>> > elsewhere!! Anyway, I think it is hard to beleive a GC is taking 2:30
>> > minutes.
>> >
>> > Abs,
>> > Pablo
>> >
>> >
>> > On 03/10/2013 04:06 PM, Sreepathi wrote:
>> >
>> >> Hi Stack/Ted/Pablo,
>> >>
>> >> Should we increase the hbase.rpc.timeout property to 5 minutes (
>> 30 ms
>> >> )  ?
>> >>
>> >> Regards,
>> >> - Sreepathi
>> >>
>> >> On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:
>> >>
>> >>  That combo should be fine.
>> 
>> >>> Great!!
>> >>>
>> >>>
>> >>>  If JVM is full GC'ing, the application is stopped.
>>  The below does not look like a full GC but that is a long pause in
>>  system
>>  time, enough to kill your zk session.
>> 
>> >>> Exactly. This pause is really making the zk expire the RS which
>> shutsdown
>> >>> (logs
>> >>> in the end of the email).
>> >>> But the question is: what is causing this pause??!!
>> >>>
>> >>>  You swapping?
>> 
>> >>> I don't think so (stats below).
>> >>>
>> >>>  Hardware is good?
>> 
>> >>> Yes, it is a 16 processor machine with 74GB of RAM and plenty disk
>> space.
>> >>> Below are some metrics I have heard about. Hope it helps.
>> >>>
>> >>>
>> >>> ** I am having some problems with the datanodes[1] which are having
>> >>> trouble to
>> >>> write. I really think the issues are related, but cannot solve any of
>> >>> them
>> >>> :(
>> >>>
>> >>> Thanks again,
>> >>> Pablo
>> >>>
>> >>> [1] http://mail-archives.apache.
>> org/mod_mbox/hadoop-hdfs-user/
>> >>> 201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr
>> >>> qfv...@mail.gmail.com%3E> >>> mod_mbox/hadoop-hdfs-user/**201303.mbox/%3CCAJzooYfS-F1KS+**
>> >>> jGOPUt15PwFjcCSzigE0APeM9FXaCr**qfv...@mail.gmail.com%3E<
>> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201303.mbox/%3ccajzooyfs-f1ks+jgoput15pwfjccszige0apem9fxacrqfv...@mail.gmail.com%3E
>> >
>> >>> >
>> >>>
>> >>> top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34,
>> 2.55,
>> >>> 1.28
>> >>> Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
>> >>> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
>> >>>   0.0%st
>> >>> Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k
>> buffers
>> >>> Swap: 51609592k total,   128312k used, 51481280k free,  1353400k
>> cached
>> >>>
>> >>> ]$ vmstat -w
>> >>> procs ---memory-- ---swap--
>> >>> -io
>> >>> --system-- -cpu---
>> >>>   r  b   swpd   free   buff  cache   si   sobi bo
>> >>> in
>> >>>cs  us sy  id wa st
>> >>>   2  0 128312   324169285838288504356000   202 53
>> >>>  0
>> >>> 0   2  1  96  1  0
>> >>>
>> >>> ]$ sar
>> >>> 02:20:01 PM all 26.18  0.00  2.90  0.63 0.00
>> >>> 70.29
>> >>> 02:30:01 PM all  1.66  0.00  1.25  1.05 0.00
>> >>> 96.04
>> >>> 02:40:01 PM all 10.01  0.00  2.14  0.75 0.00
>> >>> 87.11
>> >>> 02:50:01 PM all  0.76  0.00  0.80  1.03 0.00
>> >>> 97.40
>> >>> 03:00:01 PM all  0.23  0.00  0.30  0.71 0.00
>> >>> 98.76
>> >>> 03:10:01 PM all  0.22  0.00  0.30  0.66 0.00
>> >>> 98.82
>> >>> 03:20:01 PM all  0.22  0.00  0.31  0.76 0.00
>> >>> 98.71
>> >>> 03:30:01 PM all  0.24  0.00  0.31  0.64 0.00
>> >>> 98.81
>> >>> 03:40:01 PM all  1.13  0.00  2.9

Re: RegionServers Crashing every hour in production env

[Azuryy Yu]

Hi Pablo,
It'a terrible for a long minor GC. I don't think there are swaping from
your vmstat log.
but I just suggest you
1) add following JVM options:
-XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:GCTimeRatio=19
-XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=2
-XX:MaxTenuringThreshold=3 -XX:+UseFastAccessorMethods

2) -Xmn is two small, your total Mem is 74GB, just make -Xmn2g
3) what are you doing during long GC happened? read or write? if reading,
what the block cache size?




On Mon, Mar 11, 2013 at 6:41 AM, Stack  wrote:

> You could increase your zookeeper session timeout to 5 minutes while you
> are figuring why these long pauses.
> http://hbase.apache.org/book.html#zookeeper.session.timeout
>
> Above, there is an outage for almost 5 minutes:
>
> >> We slept 225100ms instead of 3000ms, this is likely due to a long
>
> You have ganglia or tsdb running?  When you see the big pause above, can
> you see anything going on on the machine?  (swap, iowait, concurrent fat
> mapreduce job?)
>
> St.Ack
>
>
>
> On Sun, Mar 10, 2013 at 3:29 PM, Pablo Musa  wrote:
>
> > Hi Sreepathi,
> > they say in the book (or the site), we could try it to see if it is
> really
> > a timeout error
> > or there is something more. But it is not recomended for production
> > environments.
> >
> > I could give it a try if five minutes will ensure to us that the problem
> > is the GC or
> > elsewhere!! Anyway, I think it is hard to beleive a GC is taking 2:30
> > minutes.
> >
> > Abs,
> > Pablo
> >
> >
> > On 03/10/2013 04:06 PM, Sreepathi wrote:
> >
> >> Hi Stack/Ted/Pablo,
> >>
> >> Should we increase the hbase.rpc.timeout property to 5 minutes ( 30
> ms
> >> )  ?
> >>
> >> Regards,
> >> - Sreepathi
> >>
> >> On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:
> >>
> >>  That combo should be fine.
> 
> >>> Great!!
> >>>
> >>>
> >>>  If JVM is full GC'ing, the application is stopped.
>  The below does not look like a full GC but that is a long pause in
>  system
>  time, enough to kill your zk session.
> 
> >>> Exactly. This pause is really making the zk expire the RS which
> shutsdown
> >>> (logs
> >>> in the end of the email).
> >>> But the question is: what is causing this pause??!!
> >>>
> >>>  You swapping?
> 
> >>> I don't think so (stats below).
> >>>
> >>>  Hardware is good?
> 
> >>> Yes, it is a 16 processor machine with 74GB of RAM and plenty disk
> space.
> >>> Below are some metrics I have heard about. Hope it helps.
> >>>
> >>>
> >>> ** I am having some problems with the datanodes[1] which are having
> >>> trouble to
> >>> write. I really think the issues are related, but cannot solve any of
> >>> them
> >>> :(
> >>>
> >>> Thanks again,
> >>> Pablo
> >>>
> >>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/
> >>> 201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr
> >>> qfv...@mail.gmail.com%3E >>> mod_mbox/hadoop-hdfs-user/**201303.mbox/%3CCAJzooYfS-F1KS+**
> >>> jGOPUt15PwFjcCSzigE0APeM9FXaCr**qfv...@mail.gmail.com%3E<
> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201303.mbox/%3ccajzooyfs-f1ks+jgoput15pwfjccszige0apem9fxacrqfv...@mail.gmail.com%3E
> >
> >>> >
> >>>
> >>> top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34, 2.55,
> >>> 1.28
> >>> Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
> >>> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
> >>>   0.0%st
> >>> Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k
> buffers
> >>> Swap: 51609592k total,   128312k used, 51481280k free,  1353400k cached
> >>>
> >>> ]$ vmstat -w
> >>> procs ---memory-- ---swap--
> >>> -io
> >>> --system-- -cpu---
> >>>   r  b   swpd   free   buff  cache   si   sobi bo
> >>> in
> >>>cs  us sy  id wa st
> >>>   2  0 128312   324169285838288504356000   202 53
> >>>  0
> >>> 0   2  1  96  1  0
> >>>
> >>> ]$ sar
> >>> 02:20:01 PM all 26.18  0.00  2.90  0.63 0.00
> >>> 70.29
> >>> 02:30:01 PM all  1.66  0.00  1.25  1.05 0.00
> >>> 96.04
> >>> 02:40:01 PM all 10.01  0.00  2.14  0.75 0.00
> >>> 87.11
> >>> 02:50:01 PM all  0.76  0.00  0.80  1.03 0.00
> >>> 97.40
> >>> 03:00:01 PM all  0.23  0.00  0.30  0.71 0.00
> >>> 98.76
> >>> 03:10:01 PM all  0.22  0.00  0.30  0.66 0.00
> >>> 98.82
> >>> 03:20:01 PM all  0.22  0.00  0.31  0.76 0.00
> >>> 98.71
> >>> 03:30:01 PM all  0.24  0.00  0.31  0.64 0.00
> >>> 98.81
> >>> 03:40:01 PM all  1.13  0.00  2.97  1.18 0.00
> >>> 94.73
> >>> Average:all  3.86  0.00  1.38  0.88 0.00
> >>> 93.87
> >>>
> >>> ]$ iostat
> >>> Linux 2.6.32-220.7.1.el6.x86_64 (PSLBHDN002) 03/10/2013 _x86_64_
> >>>   (16 CPU)
> >>>
> >>> avg-cpu:  %user   %nice %syste

RE: How HBase perform per-column scan?

[Liu, Raymond]

Just curious, won't ROWCOL bloom filter works for this case?

Best Regards,
Raymond Liu

> 
> As per the above said, you will need a full table scan on that CF.
> As Ted said, consider having a look at your schema design.
> 
> -Anoop-
> 
> 
> On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu  wrote:
> 
> > bq. physically column family should be able to perform efficiently
> > (storage layer
> >
> > When you scan a row, data for different column families would be
> > brought into memory (if you don't utilize HBASE-5416) Take a look at:
> >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > 1258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabp
> > anel#comment-13541258
> >
> > which was based on the settings described in:
> >
> >
> >
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=1354
> > 1191&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabp
> > anel#comment-13541191
> >
> > This boils down to your schema design. If possible, consider
> > extracting column C into its own column family.
> >
> > Cheers
> >
> > On Sun, Mar 10, 2013 at 7:14 AM, PG  wrote:
> >
> > > Hi, Ted and Anoop, thanks for your notes.
> > > I am talking about column rather than column family, since
> > > physically column family should be able to perform efficiently
> > > (storage layer, CF's are stored separately). But columns of the same
> > > column family may be
> > mixed
> > > physically, and that makes filters column value hard... So I want to
> > > know if there are any mechanism in HBase worked on this...
> > > Regards,
> > > Yun
> > >
> > > On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:
> > >
> > > > Hi, Yun:
> > > > Take a look at HBASE-5416 (Improve performance of scans with some
> > > > kind
> > of
> > > > filters) which is in 0.94.5 release.
> > > >
> > > > In your case, you can use a filter which specifies column C as the
> > > > essential family.
> > > > Here I interpret column C as column family.
> > > >
> > > > Cheers
> > > >
> > > > On Sat, Mar 9, 2013 at 11:11 AM, yun peng 
> > wrote:
> > > >
> > > >> Hi, All,
> > > >> I want to find all existing values for a given column in a HBase,
> > > >> and
> > > would
> > > >> that result in a full-table scan in HBase? For example, given a
> > > >> column
> > > C,
> > > >> the table is of very large number of rows, from which few rows
> > > >> (say
> > > only 1
> > > >> row) have non-empty values for column C. Would HBase still ues a
> > > >> full
> > > table
> > > >> scan to find this row? Or HBase has any optimization work for
> > > >> this
> > kind
> > > of
> > > >> query?
> > > >> Thanks...
> > > >> Regards
> > > >> Yun
> > > >>
> > >
> >

Re: Welcome our newest Committer Anoop

[Andrew Purtell]

Congratulations Anoop. Welcome!


On Mon, Mar 11, 2013 at 12:42 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
>
> He has contributed features related to Endpoints and CPs.
>
> Welcome Anoop and best wishes for your future work.
>
> Hope to see your continuing efforts to the community.
>
> Regards
> Ram
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: Welcome our newest Committer Anoop

[Stack]

Good on you Anoop!
St.Ack


On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
>
> He has contributed features related to Endpoints and CPs.
>
> Welcome Anoop and best wishes for your future work.
>
> Hope to see your continuing efforts to the community.
>
> Regards
> Ram
>

Re: RegionServers Crashing every hour in production env

[Stack]

You could increase your zookeeper session timeout to 5 minutes while you
are figuring why these long pauses.
http://hbase.apache.org/book.html#zookeeper.session.timeout

Above, there is an outage for almost 5 minutes:

>> We slept 225100ms instead of 3000ms, this is likely due to a long

You have ganglia or tsdb running?  When you see the big pause above, can
you see anything going on on the machine?  (swap, iowait, concurrent fat
mapreduce job?)

St.Ack



On Sun, Mar 10, 2013 at 3:29 PM, Pablo Musa  wrote:

> Hi Sreepathi,
> they say in the book (or the site), we could try it to see if it is really
> a timeout error
> or there is something more. But it is not recomended for production
> environments.
>
> I could give it a try if five minutes will ensure to us that the problem
> is the GC or
> elsewhere!! Anyway, I think it is hard to beleive a GC is taking 2:30
> minutes.
>
> Abs,
> Pablo
>
>
> On 03/10/2013 04:06 PM, Sreepathi wrote:
>
>> Hi Stack/Ted/Pablo,
>>
>> Should we increase the hbase.rpc.timeout property to 5 minutes ( 30 ms
>> )  ?
>>
>> Regards,
>> - Sreepathi
>>
>> On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:
>>
>>  That combo should be fine.

>>> Great!!
>>>
>>>
>>>  If JVM is full GC'ing, the application is stopped.
 The below does not look like a full GC but that is a long pause in
 system
 time, enough to kill your zk session.

>>> Exactly. This pause is really making the zk expire the RS which shutsdown
>>> (logs
>>> in the end of the email).
>>> But the question is: what is causing this pause??!!
>>>
>>>  You swapping?

>>> I don't think so (stats below).
>>>
>>>  Hardware is good?

>>> Yes, it is a 16 processor machine with 74GB of RAM and plenty disk space.
>>> Below are some metrics I have heard about. Hope it helps.
>>>
>>>
>>> ** I am having some problems with the datanodes[1] which are having
>>> trouble to
>>> write. I really think the issues are related, but cannot solve any of
>>> them
>>> :(
>>>
>>> Thanks again,
>>> Pablo
>>>
>>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/
>>> 201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr
>>> qfv...@mail.gmail.com%3E>> mod_mbox/hadoop-hdfs-user/**201303.mbox/%3CCAJzooYfS-F1KS+**
>>> jGOPUt15PwFjcCSzigE0APeM9FXaCr**qfv...@mail.gmail.com%3E
>>> >
>>>
>>> top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34, 2.55,
>>> 1.28
>>> Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
>>> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
>>>   0.0%st
>>> Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k buffers
>>> Swap: 51609592k total,   128312k used, 51481280k free,  1353400k cached
>>>
>>> ]$ vmstat -w
>>> procs ---memory-- ---swap--
>>> -io
>>> --system-- -cpu---
>>>   r  b   swpd   free   buff  cache   si   sobi bo
>>> in
>>>cs  us sy  id wa st
>>>   2  0 128312   324169285838288504356000   202 53
>>>  0
>>> 0   2  1  96  1  0
>>>
>>> ]$ sar
>>> 02:20:01 PM all 26.18  0.00  2.90  0.63 0.00
>>> 70.29
>>> 02:30:01 PM all  1.66  0.00  1.25  1.05 0.00
>>> 96.04
>>> 02:40:01 PM all 10.01  0.00  2.14  0.75 0.00
>>> 87.11
>>> 02:50:01 PM all  0.76  0.00  0.80  1.03 0.00
>>> 97.40
>>> 03:00:01 PM all  0.23  0.00  0.30  0.71 0.00
>>> 98.76
>>> 03:10:01 PM all  0.22  0.00  0.30  0.66 0.00
>>> 98.82
>>> 03:20:01 PM all  0.22  0.00  0.31  0.76 0.00
>>> 98.71
>>> 03:30:01 PM all  0.24  0.00  0.31  0.64 0.00
>>> 98.81
>>> 03:40:01 PM all  1.13  0.00  2.97  1.18 0.00
>>> 94.73
>>> Average:all  3.86  0.00  1.38  0.88 0.00
>>> 93.87
>>>
>>> ]$ iostat
>>> Linux 2.6.32-220.7.1.el6.x86_64 (PSLBHDN002) 03/10/2013 _x86_64_
>>>   (16 CPU)
>>>
>>> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>>> 1.860.000.960.780.00   96.41
>>>
>>> Device:tps   Blk_read/s   Blk_wrtn/s   Blk_read Blk_wrtn
>>> sda   1.2320.2623.53  521533196 605566924
>>> sdb   6.51   921.55   241.90 23717850730 6225863488
>>> sdc   6.22   921.83   236.41 23725181162 6084471192
>>> sdd   6.25   925.13   237.26 23810004970 6106357880
>>> sde   6.19   913.90   235.60 23521108818 6063722504
>>> sdh   6.26   933.08   237.77 24014594546 6119511376
>>> sdg   6.18   914.36   235.31 23532747378 6056257016
>>> sdf   6.24   923.66   235.33 23772251810 6056604008
>>>
>>> Some more logging which reinforce

Re: RegionServers Crashing every hour in production env

[Pablo Musa]

Hi Sreepathi,
they say in the book (or the site), we could try it to see if it is 
really a timeout error
or there is something more. But it is not recomended for production 
environments.


I could give it a try if five minutes will ensure to us that the problem 
is the GC or
elsewhere!! Anyway, I think it is hard to beleive a GC is taking 2:30 
minutes.


Abs,
Pablo

On 03/10/2013 04:06 PM, Sreepathi wrote:

Hi Stack/Ted/Pablo,

Should we increase the hbase.rpc.timeout property to 5 minutes ( 30 ms
)  ?

Regards,
- Sreepathi

On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:


That combo should be fine.

Great!!



If JVM is full GC'ing, the application is stopped.
The below does not look like a full GC but that is a long pause in system
time, enough to kill your zk session.

Exactly. This pause is really making the zk expire the RS which shutsdown
(logs
in the end of the email).
But the question is: what is causing this pause??!!


You swapping?

I don't think so (stats below).


Hardware is good?

Yes, it is a 16 processor machine with 74GB of RAM and plenty disk space.
Below are some metrics I have heard about. Hope it helps.


** I am having some problems with the datanodes[1] which are having
trouble to
write. I really think the issues are related, but cannot solve any of them
:(

Thanks again,
Pablo

[1] http://mail-archives.apache.**org/mod_mbox/hadoop-hdfs-user/**
201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr**
qfv...@mail.gmail.com%3E

top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34, 2.55,
1.28
Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
  0.0%st
Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k buffers
Swap: 51609592k total,   128312k used, 51481280k free,  1353400k cached

]$ vmstat -w
procs ---memory-**- ---swap-- -io
--system-- -cpu---
  r  b   swpd   free   buff  cache   si   sobi bo   in
   cs  us sy  id wa st
  2  0 128312   324169285838288504356000   202 530
0   2  1  96  1  0

]$ sar
02:20:01 PM all 26.18  0.00  2.90  0.63 0.00 70.29
02:30:01 PM all  1.66  0.00  1.25  1.05 0.00 96.04
02:40:01 PM all 10.01  0.00  2.14  0.75 0.00 87.11
02:50:01 PM all  0.76  0.00  0.80  1.03 0.00 97.40
03:00:01 PM all  0.23  0.00  0.30  0.71 0.00 98.76
03:10:01 PM all  0.22  0.00  0.30  0.66 0.00 98.82
03:20:01 PM all  0.22  0.00  0.31  0.76 0.00 98.71
03:30:01 PM all  0.24  0.00  0.31  0.64 0.00 98.81
03:40:01 PM all  1.13  0.00  2.97  1.18 0.00 94.73
Average:all  3.86  0.00  1.38  0.88 0.00 93.87

]$ iostat
Linux 2.6.32-220.7.1.el6.x86_64 (PSLBHDN002) 03/10/2013 _x86_64_
  (16 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
1.860.000.960.780.00   96.41

Device:tps   Blk_read/s   Blk_wrtn/s   Blk_read Blk_wrtn
sda   1.2320.2623.53  521533196 605566924
sdb   6.51   921.55   241.90 23717850730 6225863488
sdc   6.22   921.83   236.41 23725181162 6084471192
sdd   6.25   925.13   237.26 23810004970 6106357880
sde   6.19   913.90   235.60 23521108818 6063722504
sdh   6.26   933.08   237.77 24014594546 6119511376
sdg   6.18   914.36   235.31 23532747378 6056257016
sdf   6.24   923.66   235.33 23772251810 6056604008

Some more logging which reinforce that the RS crash is happening because of
timeout. However this time the GC log is not accusing a big time.

#RS LOG#
2013-03-10 15:37:46,712 INFO org.apache.zookeeper.**ClientCnxn: Client
session timed out, have not heard from server in 257739ms for sessionid
0x13d3c4bcba6014a, closing socket connection and attempting reconnect
2013-03-10 15:37:46,712 INFO org.apache.zookeeper.**ClientCnxn: Client
session timed out, have not heard from server in 226785ms for sessionid
0x13d3c4bcba60149, closing socket connection and attempting reconnect
2013-03-10 15:37:46,712 DEBUG org.apache.hadoop.hbase.io.**hfile.LruBlockCache:
Stats: total=61.91 MB, free=1.94 GB, max=2 GB, blocks=1254, accesses=60087,
hits=58811, hitRatio=97.87%, , cachingAccesses=60069, cachingHits=58811,
cachingHitsRatio=97.90%, , evictions=0, evicted=0, evictedPerRun=NaN
2013-03-10 15:37:46,712 WARN org.apache.hadoop.hbase.util.**Sleeper: We
slept 225100ms instead of 3000ms, this is likely due to a long garbage
collecting pause and it's usually bad, see 

Re: unsubscribe request

[lars hofhansl]

No problem, Dan.




 From: Dan Han 
To: user@hbase.apache.org 
Sent: Sunday, March 10, 2013 12:36 PM
Subject: Re: unsubscribe request
 
Embarassing! It was supposed to send to another email address. Sorry about
this.

Best Wishes
Dan Han


On Sun, Mar 10, 2013 at 1:15 PM, Panshul Whisper wrote:

> lol
>
>
> On Sun, Mar 10, 2013 at 7:29 PM, Dan Han  wrote:
>
> > I would like to unsubscribe now as the email is huge. Thanks.
> >
> > Best Wishes
> > Dan Han
> >
>
>
>
> --
> Regards,
> Ouch Whisper
> 010101010101
>

Re: Welcome our newest Committer Anoop

[lars hofhansl]

Congrats Anoop. Welcome.
(Don't break the tests :) )




 From: ramkrishna vasudevan 
To: d...@hbase.apache.org; user@hbase.apache.org 
Sent: Sunday, March 10, 2013 9:42 AM
Subject: Welcome our newest Committer Anoop
 
Hi All

Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
great and he has helped lot of users in the mailing list.

He has contributed features related to Endpoints and CPs.

Welcome Anoop and best wishes for your future work.

Hope to see your continuing efforts to the community.

Regards
Ram

Re: unsubscribe request

[Dan Han]

Embarassing! It was supposed to send to another email address. Sorry about
this.

Best Wishes
Dan Han


On Sun, Mar 10, 2013 at 1:15 PM, Panshul Whisper wrote:

> lol
>
>
> On Sun, Mar 10, 2013 at 7:29 PM, Dan Han  wrote:
>
> > I would like to unsubscribe now as the email is huge. Thanks.
> >
> > Best Wishes
> > Dan Han
> >
>
>
>
> --
> Regards,
> Ouch Whisper
> 010101010101
>

Re: unsubscribe request

[Panshul Whisper]

lol


On Sun, Mar 10, 2013 at 7:29 PM, Dan Han  wrote:

> I would like to unsubscribe now as the email is huge. Thanks.
>
> Best Wishes
> Dan Han
>



-- 
Regards,
Ouch Whisper
010101010101

Re: RegionServers Crashing every hour in production env

[Sreepathi]

Hi Stack/Ted/Pablo,

Should we increase the hbase.rpc.timeout property to 5 minutes ( 30 ms
)  ?

Regards,
- Sreepathi

On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa  wrote:

> > That combo should be fine.
>
> Great!!
>
>
> > If JVM is full GC'ing, the application is stopped.
> > The below does not look like a full GC but that is a long pause in system
> > time, enough to kill your zk session.
>
> Exactly. This pause is really making the zk expire the RS which shutsdown
> (logs
> in the end of the email).
> But the question is: what is causing this pause??!!
>
> > You swapping?
>
> I don't think so (stats below).
>
> > Hardware is good?
>
> Yes, it is a 16 processor machine with 74GB of RAM and plenty disk space.
> Below are some metrics I have heard about. Hope it helps.
>
>
> ** I am having some problems with the datanodes[1] which are having
> trouble to
> write. I really think the issues are related, but cannot solve any of them
> :(
>
> Thanks again,
> Pablo
>
> [1] http://mail-archives.apache.**org/mod_mbox/hadoop-hdfs-user/**
> 201303.mbox/%3CCAJzooYfS-F1KS+**jGOPUt15PwFjcCSzigE0APeM9FXaCr**
> qfv...@mail.gmail.com%3E
>
> top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34, 2.55,
> 1.28
> Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,
>  0.0%st
> Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k buffers
> Swap: 51609592k total,   128312k used, 51481280k free,  1353400k cached
>
> ]$ vmstat -w
> procs ---memory-**- ---swap-- -io
> --system-- -cpu---
>  r  b   swpd   free   buff  cache   si   sobi bo   in
>   cs  us sy  id wa st
>  2  0 128312   324169285838288504356000   202 530
>0   2  1  96  1  0
>
> ]$ sar
> 02:20:01 PM all 26.18  0.00  2.90  0.63 0.00 70.29
> 02:30:01 PM all  1.66  0.00  1.25  1.05 0.00 96.04
> 02:40:01 PM all 10.01  0.00  2.14  0.75 0.00 87.11
> 02:50:01 PM all  0.76  0.00  0.80  1.03 0.00 97.40
> 03:00:01 PM all  0.23  0.00  0.30  0.71 0.00 98.76
> 03:10:01 PM all  0.22  0.00  0.30  0.66 0.00 98.82
> 03:20:01 PM all  0.22  0.00  0.31  0.76 0.00 98.71
> 03:30:01 PM all  0.24  0.00  0.31  0.64 0.00 98.81
> 03:40:01 PM all  1.13  0.00  2.97  1.18 0.00 94.73
> Average:all  3.86  0.00  1.38  0.88 0.00 93.87
>
> ]$ iostat
> Linux 2.6.32-220.7.1.el6.x86_64 (PSLBHDN002) 03/10/2013 _x86_64_
>  (16 CPU)
>
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>1.860.000.960.780.00   96.41
>
> Device:tps   Blk_read/s   Blk_wrtn/s   Blk_read Blk_wrtn
> sda   1.2320.2623.53  521533196 605566924
> sdb   6.51   921.55   241.90 23717850730 6225863488
> sdc   6.22   921.83   236.41 23725181162 6084471192
> sdd   6.25   925.13   237.26 23810004970 6106357880
> sde   6.19   913.90   235.60 23521108818 6063722504
> sdh   6.26   933.08   237.77 24014594546 6119511376
> sdg   6.18   914.36   235.31 23532747378 6056257016
> sdf   6.24   923.66   235.33 23772251810 6056604008
>
> Some more logging which reinforce that the RS crash is happening because of
> timeout. However this time the GC log is not accusing a big time.
>
> #RS LOG#
> 2013-03-10 15:37:46,712 INFO org.apache.zookeeper.**ClientCnxn: Client
> session timed out, have not heard from server in 257739ms for sessionid
> 0x13d3c4bcba6014a, closing socket connection and attempting reconnect
> 2013-03-10 15:37:46,712 INFO org.apache.zookeeper.**ClientCnxn: Client
> session timed out, have not heard from server in 226785ms for sessionid
> 0x13d3c4bcba60149, closing socket connection and attempting reconnect
> 2013-03-10 15:37:46,712 DEBUG 
> org.apache.hadoop.hbase.io.**hfile.LruBlockCache:
> Stats: total=61.91 MB, free=1.94 GB, max=2 GB, blocks=1254, accesses=60087,
> hits=58811, hitRatio=97.87%, , cachingAccesses=60069, cachingHits=58811,
> cachingHitsRatio=97.90%, , evictions=0, evicted=0, evictedPerRun=NaN
> 2013-03-10 15:37:46,712 WARN org.apache.hadoop.hbase.util.**Sleeper: We
> slept 225100ms instead of 3000ms, this is likely due to a long garbage
> collecting pause and it's usually bad, see http://hbase.apache.org/book.**
> html#trouble.rs.runtime.**zkexpired
> 2013-03-10 15:37:46,714 WARN org.apache.hadoop.hdfs.**DFSClient:
> DFSOutputStream ResponseProcesso

Re: RegionServers Crashing every hour in production env

[Pablo Musa]

> That combo should be fine.

Great!!

> If JVM is full GC'ing, the application is stopped.
> The below does not look like a full GC but that is a long pause in system
> time, enough to kill your zk session.

Exactly. This pause is really making the zk expire the RS which 
shutsdown (logs

in the end of the email).
But the question is: what is causing this pause??!!

> You swapping?

I don't think so (stats below).

> Hardware is good?

Yes, it is a 16 processor machine with 74GB of RAM and plenty disk space.
Below are some metrics I have heard about. Hope it helps.


** I am having some problems with the datanodes[1] which are having 
trouble to
write. I really think the issues are related, but cannot solve any of 
them :(


Thanks again,
Pablo

[1] 
http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201303.mbox/%3ccajzooyfs-f1ks+jgoput15pwfjccszige0apem9fxacrqfv...@mail.gmail.com%3E


top - 15:38:04 up 297 days, 21:03,  2 users,  load average: 4.34, 2.55, 1.28
Tasks: 528 total,   1 running, 527 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi, 0.0%si,  
0.0%st

Mem:  74187256k total, 29493992k used, 44693264k free,  5836576k buffers
Swap: 51609592k total,   128312k used, 51481280k free,  1353400k cached

]$ vmstat -w
procs ---memory-- ---swap-- -io 
--system-- -cpu---
 r  b   swpd   free   buff  cache   si   sobi bo   
in   cs  us sy  id wa st
 2  0 128312   324169285838288504356000   202 53
00   2  1  96  1  0


]$ sar
02:20:01 PM all 26.18  0.00  2.90  0.63 0.00 70.29
02:30:01 PM all  1.66  0.00  1.25  1.05 0.00 96.04
02:40:01 PM all 10.01  0.00  2.14  0.75 0.00 87.11
02:50:01 PM all  0.76  0.00  0.80  1.03 0.00 97.40
03:00:01 PM all  0.23  0.00  0.30  0.71 0.00 98.76
03:10:01 PM all  0.22  0.00  0.30  0.66 0.00 98.82
03:20:01 PM all  0.22  0.00  0.31  0.76 0.00 98.71
03:30:01 PM all  0.24  0.00  0.31  0.64 0.00 98.81
03:40:01 PM all  1.13  0.00  2.97  1.18 0.00 94.73
Average:all  3.86  0.00  1.38  0.88 0.00 93.87

]$ iostat
Linux 2.6.32-220.7.1.el6.x86_64 (PSLBHDN002) 03/10/2013 _x86_64_
(16 CPU)


avg-cpu:  %user   %nice %system %iowait  %steal   %idle
   1.860.000.960.780.00   96.41

Device:tps   Blk_read/s   Blk_wrtn/s   Blk_read Blk_wrtn
sda   1.2320.2623.53  521533196 605566924
sdb   6.51   921.55   241.90 23717850730 6225863488
sdc   6.22   921.83   236.41 23725181162 6084471192
sdd   6.25   925.13   237.26 23810004970 6106357880
sde   6.19   913.90   235.60 23521108818 6063722504
sdh   6.26   933.08   237.77 24014594546 6119511376
sdg   6.18   914.36   235.31 23532747378 6056257016
sdf   6.24   923.66   235.33 23772251810 6056604008

Some more logging which reinforce that the RS crash is happening because of
timeout. However this time the GC log is not accusing a big time.

#RS LOG#
2013-03-10 15:37:46,712 INFO org.apache.zookeeper.ClientCnxn: Client 
session timed out, have not heard from server in 257739ms for sessionid 
0x13d3c4bcba6014a, closing socket connection and attempting reconnect
2013-03-10 15:37:46,712 INFO org.apache.zookeeper.ClientCnxn: Client 
session timed out, have not heard from server in 226785ms for sessionid 
0x13d3c4bcba60149, closing socket connection and attempting reconnect
2013-03-10 15:37:46,712 DEBUG 
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Stats: total=61.91 MB, 
free=1.94 GB, max=2 GB, blocks=1254, accesses=60087, hits=58811, 
hitRatio=97.87%, , cachingAccesses=60069, cachingHits=58811, 
cachingHitsRatio=97.90%, , evictions=0, evicted=0, evictedPerRun=NaN
2013-03-10 15:37:46,712 WARN org.apache.hadoop.hbase.util.Sleeper: We 
slept 225100ms instead of 3000ms, this is likely due to a long garbage 
collecting pause and it's usually bad, see 
http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2013-03-10 15:37:46,714 WARN org.apache.hadoop.hdfs.DFSClient: 
DFSOutputStream ResponseProcessor exception  for block 
BP-43236042-172.17.2.10-1362490844340:blk_-6834190810033122569_25150229

java.io.EOFException: Premature EOF: no length prefix available
at 
org.apache.hadoop.hdfs.protocol.HdfsProtoUtil.vintPrefixed(HdfsProtoUtil.java:171)
at 
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:114)
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:670)
2013-03-10 15:37:46,716 ERROR 
org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoo

Re: can we use same column name for 2 different column families?

[shashwat shriparv]

Ya sure you can have that... as you always specify columnfamily to read out
of hbase, i dont think there should be a problem in that.



∞
Shashwat Shriparv



On Sun, Mar 10, 2013 at 11:41 PM, Ramasubramanian Narayanan <
ramasubramanian.naraya...@gmail.com> wrote:

> Hi,
>
> Is it fine to use same column name for 2 different column families?
>
> For example,
>
> In a table "emp",
>
> can we have column name "dob" under column family "F1" & "F2"?
>
> Please let me know the impact of having like this if any...
>
> Note : I don't want to use dob1 or some other field name for the second
> column... use case is like that...
>
> regards,
> Rams
>

can we use same column name for 2 different column families?

[Ramasubramanian Narayanan]

Hi,

Is it fine to use same column name for 2 different column families?

For example,

In a table "emp",

can we have column name "dob" under column family "F1" & "F2"?

Please let me know the impact of having like this if any...

Note : I don't want to use dob1 or some other field name for the second
column... use case is like that...

regards,
Rams

Re: Welcome our newest Committer Anoop

[Jesse Yates]

Great job Anoop! Keep up the good work.

- Jesse Yates

Sent from my iPhone

On Mar 10, 2013, at 9:42 AM, ramkrishna vasudevan 
 wrote:

> Hi All
> 
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
> 
> He has contributed features related to Endpoints and CPs.
> 
> Welcome Anoop and best wishes for your future work.
> 
> Hope to see your continuing efforts to the community.
> 
> Regards
> Ram

Re: Welcome our newest Committer Anoop

[Anil Gupta]

Congrats & Welcome Anoop!!

Best Regards,
Anil

On Mar 10, 2013, at 9:58 AM, Ted Yu  wrote:

> Congratulations, Anoop.
> 
> Keep up the good work.
> 
> On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan <
> ramkrishna.s.vasude...@gmail.com> wrote:
> 
>> Hi All
>> 
>> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
>> great and he has helped lot of users in the mailing list.
>> 
>> He has contributed features related to Endpoints and CPs.
>> 
>> Welcome Anoop and best wishes for your future work.
>> 
>> Hope to see your continuing efforts to the community.
>> 
>> Regards
>> Ram
>> 

Re: Welcome our newest Committer Anoop

[Ted Yu]

Congratulations, Anoop.

Keep up the good work.

On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Pls welcome Anoop, our newest committer.  Anoop's work in HBase has been
> great and he has helped lot of users in the mailing list.
>
> He has contributed features related to Endpoints and CPs.
>
> Welcome Anoop and best wishes for your future work.
>
> Hope to see your continuing efforts to the community.
>
> Regards
> Ram
>

Re: How HBase perform per-column scan?

[Anoop John]

As per the above said, you will need a full table scan on that CF.
As Ted said, consider having a look at your schema design.

-Anoop-


On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu  wrote:

> bq. physically column family should be able to perform efficiently (storage
> layer
>
> When you scan a row, data for different column families would be brought
> into memory (if you don't utilize HBASE-5416)
> Take a look at:
>
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13541258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13541258
>
> which was based on the settings described in:
>
>
> https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13541191&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13541191
>
> This boils down to your schema design. If possible, consider extracting
> column C into its own column family.
>
> Cheers
>
> On Sun, Mar 10, 2013 at 7:14 AM, PG  wrote:
>
> > Hi, Ted and Anoop, thanks for your notes.
> > I am talking about column rather than column family, since physically
> > column family should be able to perform efficiently (storage layer, CF's
> > are stored separately). But columns of the same column family may be
> mixed
> > physically, and that makes filters column value hard... So I want to know
> > if there are any mechanism in HBase worked on this...
> > Regards,
> > Yun
> >
> > On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:
> >
> > > Hi, Yun:
> > > Take a look at HBASE-5416 (Improve performance of scans with some kind
> of
> > > filters) which is in 0.94.5 release.
> > >
> > > In your case, you can use a filter which specifies column C as the
> > > essential family.
> > > Here I interpret column C as column family.
> > >
> > > Cheers
> > >
> > > On Sat, Mar 9, 2013 at 11:11 AM, yun peng 
> wrote:
> > >
> > >> Hi, All,
> > >> I want to find all existing values for a given column in a HBase, and
> > would
> > >> that result in a full-table scan in HBase? For example, given a column
> > C,
> > >> the table is of very large number of rows, from which few rows (say
> > only 1
> > >> row) have non-empty values for column C. Would HBase still ues a full
> > table
> > >> scan to find this row? Or HBase has any optimization work for this
> kind
> > of
> > >> query?
> > >> Thanks...
> > >> Regards
> > >> Yun
> > >>
> >
>

Re: How HBase perform per-column scan?

[Ted Yu]

bq. physically column family should be able to perform efficiently (storage
layer

When you scan a row, data for different column families would be brought
into memory (if you don't utilize HBASE-5416)
Take a look at:
https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13541258&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13541258

which was based on the settings described in:

https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13541191&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13541191

This boils down to your schema design. If possible, consider extracting
column C into its own column family.

Cheers

On Sun, Mar 10, 2013 at 7:14 AM, PG  wrote:

> Hi, Ted and Anoop, thanks for your notes.
> I am talking about column rather than column family, since physically
> column family should be able to perform efficiently (storage layer, CF's
> are stored separately). But columns of the same column family may be mixed
> physically, and that makes filters column value hard... So I want to know
> if there are any mechanism in HBase worked on this...
> Regards,
> Yun
>
> On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:
>
> > Hi, Yun:
> > Take a look at HBASE-5416 (Improve performance of scans with some kind of
> > filters) which is in 0.94.5 release.
> >
> > In your case, you can use a filter which specifies column C as the
> > essential family.
> > Here I interpret column C as column family.
> >
> > Cheers
> >
> > On Sat, Mar 9, 2013 at 11:11 AM, yun peng  wrote:
> >
> >> Hi, All,
> >> I want to find all existing values for a given column in a HBase, and
> would
> >> that result in a full-table scan in HBase? For example, given a column
> C,
> >> the table is of very large number of rows, from which few rows (say
> only 1
> >> row) have non-empty values for column C. Would HBase still ues a full
> table
> >> scan to find this row? Or HBase has any optimization work for this kind
> of
> >> query?
> >> Thanks...
> >> Regards
> >> Yun
> >>
>

Re: How HBase perform per-column scan?

[PG]

Hi, Ted and Anoop, thanks for your notes.
I am talking about column rather than column family, since physically column 
family should be able to perform efficiently (storage layer, CF's are stored 
separately). But columns of the same column family may be mixed physically, and 
that makes filters column value hard... So I want to know if there are any 
mechanism in HBase worked on this...
Regards,
Yun

On Mar 10, 2013, at 10:01 AM, Ted Yu  wrote:

> Hi, Yun:
> Take a look at HBASE-5416 (Improve performance of scans with some kind of
> filters) which is in 0.94.5 release.
> 
> In your case, you can use a filter which specifies column C as the
> essential family.
> Here I interpret column C as column family.
> 
> Cheers
> 
> On Sat, Mar 9, 2013 at 11:11 AM, yun peng  wrote:
> 
>> Hi, All,
>> I want to find all existing values for a given column in a HBase, and would
>> that result in a full-table scan in HBase? For example, given a column C,
>> the table is of very large number of rows, from which few rows (say only 1
>> row) have non-empty values for column C. Would HBase still ues a full table
>> scan to find this row? Or HBase has any optimization work for this kind of
>> query?
>> Thanks...
>> Regards
>> Yun
>> 

Re: How HBase perform per-column scan?

[Ted Yu]

Hi, Yun:
Take a look at HBASE-5416 (Improve performance of scans with some kind of
filters) which is in 0.94.5 release.

In your case, you can use a filter which specifies column C as the
essential family.
Here I interpret column C as column family.

Cheers

On Sat, Mar 9, 2013 at 11:11 AM, yun peng  wrote:

> Hi, All,
> I want to find all existing values for a given column in a HBase, and would
> that result in a full-table scan in HBase? For example, given a column C,
> the table is of very large number of rows, from which few rows (say only 1
> row) have non-empty values for column C. Would HBase still ues a full table
> scan to find this row? Or HBase has any optimization work for this kind of
> query?
> Thanks...
> Regards
> Yun
>