RE: HBaseCon Plans?

[Bijieshan]

For HBaseCon Asia 2018, we(Huawei) can host it in Beijing this year:-)

Jieshan.
-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 2018年2月9日 2:08
To: HBase Dev List <dev@hbase.apache.org>
Subject: Re: HBaseCon Plans?

On Thu, Feb 8, 2018 at 9:37 AM, Andrew Purtell <andrew.purt...@gmail.com>
wrote:

> This is Lars H. on Andy's phone.
>
> I'm looking into hosting HBaseCon at Salesforce this year. Hopefully 
> in the new Salesforce tower. Gimme a few days perhaps a week.
>
Thanks.
>
>
Only if it is the top floor Lars. Otherwise, HBaseConEastBay -- Tacubaya, 
Starline Social Club -- here we come (smile).
St.Ack





> -- Lars
>
> > On Feb 8, 2018, at 4:18 AM, Jean-Marc Spaggiari 
> > <jean-m...@spaggiari.org>
> wrote:
> >
> > So who's jumping in for NY or SF? ;)
> >
> > 2018-02-08 4:46 GMT-05:00 Bijieshan <bijies...@huawei.com>:
> >
> >> Huawei can continue to hold HBaseCon Asia 2018 :-)
> >>
> >> Best Regards,
> >> Jieshan.
> >> -Original Message-
> >> From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of
> Stack
> >> Sent: 2018年2月8日 0:37
> >> To: HBase Dev List <dev@hbase.apache.org>
> >> Subject: Re: HBaseCon Plans?
> >>
> >>> On Fri, Feb 2, 2018 at 9:13 PM, Mike Drob <mad...@cloudera.com> wrote:
> >>>
> >>> Hi folks, has there been any consideration put forth toward the 
> >>> next HBaseCon? The last one was very productive for me personally, 
> >>> but I hadn't heard anything about the schedule for 2018 so figured 
> >>> I could
> ask
> >> on list.
> >>>
> >>> Mike
> >>>
> >>
> >> Is been kinda quiet this year in terms of hbasecon2018. We, the
> community,
> >> have been running the last bunch hosted by a generous, main sponsor
> (Huawei
> >> in Shenzhen and Google on east and west coast). If there was the
> interest,
> >> we could go beat the bushes to turn up a venue and a date. Wouldn't
> have to
> >> be a grand affair.
> >>
> >> Thanks,
> >> St.Ack
> >>
>

RE: HBaseCon Plans?

[Bijieshan]

Huawei can continue to hold HBaseCon Asia 2018 :-)

Best Regards,
Jieshan.
-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 2018年2月8日 0:37
To: HBase Dev List 
Subject: Re: HBaseCon Plans?

On Fri, Feb 2, 2018 at 9:13 PM, Mike Drob  wrote:

> Hi folks, has there been any consideration put forth toward the next 
> HBaseCon? The last one was very productive for me personally, but I 
> hadn't heard anything about the schedule for 2018 so figured I could ask on 
> list.
>
> Mike
>

Is been kinda quiet this year in terms of hbasecon2018. We, the community, have 
been running the last bunch hosted by a generous, main sponsor (Huawei in 
Shenzhen and Google on east and west coast). If there was the interest, we 
could go beat the bushes to turn up a venue and a date. Wouldn't have to be a 
grand affair.

Thanks,
St.Ack

RE: hbaseconasia2017 slides and a few phtotos

[Bijieshan]

Thanks for uploading all the slides and the good write-up, stackHope more 
people will join us next year:)

Jieshan.
-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 2017年8月7日 23:12
To: HBase Dev List ; Hbase-User 
Subject: hbaseconasia2017 slides and a few phtotos

We had a nice day out in Shenzhen at HBaseCon Asia last friday [0] (August 
4th). There were a bunch of great talks [1] by the likes of jd.com, huawei, 
xiaomi, alibaba and others. Those given in Chinese had best attendance. A good 
few folks graciously had slides in English for the language-dumb (like
myself) while their talk native.

A couple of hbase-as-a-service in the cloud are coming down the pipe, a few 
presenters talked about being at current scale limits, China Life keeps all 
data as JSON in hbase instances, and there was an interesting talk on upping 
utilization by deploying hbase with kubernetes (>1 container per node).

Best quote: "HBase is just a kid--it is only ten years old" from our Yu Li 
talking about interesting write speedups coming from Alibaba arguing there are 
many more speedups to be had.

I attached a few pictures. I'll put up more after I fixup the hbasecon home 
page and redirect later.

The day after, the 5th, there was a dev meetup at the Huawei office; notes to 
follow.

Thanks again to the program committee, the sponsors, and to our gracious host 
Huawei. Jieshan in particular did an amazing job running the show taking care 
of speakers.

St.Ack
0.  Click on 'view details' on the right hand side of this page for agenda 
(eventbrite won't show the event page at the end of the link
post-the-event): https://www.eventbrite.com/e/hbasecon-asia-2017-
tickets-34935546159#
1. Slides up on slideshare: https://www.slideshare.net/ 
search/slideshow?searchfrom=header=hbaseconasia2017=
any=all=en=

​
 _YP99027.JPG

​
The program committee taking questions at the end of the day
​
 _YP99181.JPG

​
This one is of all the speakers
​
 _YP99193.JPG

​

HBaseCon Asia 2017 Registration is now open

[Bijieshan]

HBaseCon Asia 2017 Registration is now open. The below is the registration site:
http://hbaseconasia.eventbrite.com

The registration is free.

If you want to be a speaker, please submit your abstract from here:
https://easychair.org/cfp/HBaseConAsia2017-

Any doubts or help for attending HBaseCon Asia 2017, please send email to me: 
bijies...@huawei.com<mailto:bijies...@huawei.com>

Bijieshan.

RE: ANNOUNCE: Yu Li joins the Apache HBase PMC

[Bijieshan]

Congratulations!

Jieshan.
-Original Message-
From: Anoop John [mailto:anoop.hb...@gmail.com] 
Sent: 2017年4月14日 22:22
To: dev@hbase.apache.org; u...@hbase.apache.org
Cc: Yu Li 
Subject: ANNOUNCE: Yu Li joins the Apache HBase PMC

On behalf of the Apache HBase PMC I"m pleased to announce that Yu Li has 
accepted our invitation to become a PMC member on the Apache HBase project. He 
has been an active contributor to HBase for past many years. Looking forward 
for many more contributions from him.

Welcome to the PMC, Yu Li...


-Anoop-

RE: split failed caused by FileNotFoundException

[Bijieshan]

Nice find, zhoushuaifeng:)

Suggest to raise an issue for 94.

Jieshan.

From: 周帅锋 [zhoushuaif...@gmail.com]
Sent: Thursday, December 04, 2014 6:01 PM
To: dev
Subject: Re: split failed caused by FileNotFoundException

I rechecked the code in 0.98, this problem is solved by check the store
object in the compactrunner and cance the compact the compact.
HRegion.compact:

  byte[] cf = Bytes.toBytes(store.getColumnFamilyName());
  if (stores.get(cf) != store) {
LOG.warn(Store  + store.getColumnFamilyName() +  on region  +
this
+  has been re-instantiated, cancel this compaction request. 
+  It may be caused by the roll back of split transaction);
return false;
  }


But, is it better to replease the store object by the new one and continue
the compact on the store, instead of cancel?


2014-12-04 15:00 GMT+08:00 周帅锋 zhoushuaif...@gmail.com:

 In our hbase clusters, split sometimes failed because the file to be
 splited does not exist in parent region. In 0.94.2, this will cause
 regionserver shutdown because the split transction has reached  PONR state.
 In 0.94.20 or 0.98, split will fail and can roll back, because the split
 transction only reach  the state offlined_parent.

 In 0.94.2, the error is like below:
 2014-09-23 22:27:55,710 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
 Offlined parent region x in META
 2014-09-23 22:27:55,820 INFO
 org.apache.hadoop.hbase.regionserver.SplitRequest: Running rollback/cleanup
 of failed split of x
 Caused by: java.io.IOException: java.io.IOException:
 java.io.FileNotFoundException: File does not exist: x
 Caused by: java.io.IOException: java.io.FileNotFoundException: File does
 not exist: x
 Caused by: java.io.FileNotFoundException: File does not exist: x
 2014-09-23 22:27:55,823 FATAL
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
 xxx,60020,1411383568857: Abort; we got an error after point-of-no-return

 The reasion of missing files is a little complex, the whole procedure
 include two failure split and one compact:
 1) there are too many files in the region and compact is requested, but
 not execute because there are many CompactionRequests(compactionRunners) in
 the compaction queue. The compactionRequest hodes the object of the Store,
 and also hodes a storefile list to compact of the store.

 2) the region size is big enough, and split is requested. the region is
 offline during spliting,and the store is closed. but the split failed when
 spliting files(for some reason, like io busy, etc. causing time out)
 2014-09-23 18:26:02,738 INFO
 org.apache.hadoop.hbase.regionserver.SplitRequest: Running rollback/cleanup
 of failed split of x; Took too long to split the files and create the
 references, aborting split

 3) split successfully roll back, and the region is online again. During
 roll back procedure, a new Store object is created, but the store in the
 compaction queue did not removed, so there are two(or maybe more) store
 object in regionserver.

 4) the compaction on the store of the region requested before running, and
 some storefiles are compact and removed, new bigger storefiles are created.
 but the store reinitialized in the rollback split procedure doesn't know
 the change of the storefiles.

 5) split on region running again and fail again, because the storefiles in
 parrent region doesn't exist(removed by compaction). Also, the split
 transction doesn't know that there is a new file created by the compaction.
 In 0.94.2, this error can't be found until the daughter region open, but
 it's too late, the split failed at PONR state, and this will causing
 regionserver shutdown. In 0.94.20 and 0.98, when doing splitStoreFiles, it
 will looking into the storefile in the parent region and can found the
 error before PONR, so split failure can be roll back.
  code in HRegionFileSystem.splitStoreFile:
  ...
  byte[] lastKey = f.createReader().getLastKey();

 So, this situation is a fatal error in previous 0.94 version, and also a
 common bug in the later 0.94 and higher version. And this is also the
 reason why sometimes storefile reader is null(closed by the first failure
 split).

RE: Questions about HBase replication

[Bijieshan]

But even with replicationSink, replicationSource still need to use RPC to
ship entries.
So, the potential problem you said may still occur.

Replication has its own handler, which is different from normal  handlers.
You can check it from code:)

Jieshan.

From: Cliff [cliffcheng...@gmail.com]
Sent: Monday, November 24, 2014 8:08 PM
To: hbase-...@hadoop.apache.org
Subject: RE: Questions about HBase replication

1.
But even with replicationSink, replicationSource still need to use RPC to
ship entries.
So, the potential problem you said may still occur.
What's the most important reason for using replicationSink even with two
time I/O?
Thank you.




--
View this message in context: 
http://apache-hbase.679495.n3.nabble.com/Questions-about-HBase-replication-tp4066266p4066293.html
Sent from the HBase Developer mailing list archive at Nabble.com.

RE: Questions about HBase replication

[Bijieshan]

1.
Why does HBase replication need replicationSink?
I think replicationSource can do replicationSink's work as well.
And if we don't use replicationSink, we just need one time I/O.

   ReplicationSink used to apply all HLog edits to peer cluster. If we remove 
ReplicationSink from current architecture, so how to send data to peer cluster? 
Using API HTableInterface#put?
   So there's no difference between replication write requests and normal user 
write requests. One potential problem is that the replication write request may 
occupy all the handler threads on RegionServer, then affect normal user write 
requests.

2.
The queue added HLog path in replicationSource is PriorityBlockingQueue.
If the queue is full, HLog path cannot be added to the queue.
How to deal with the situation?

   We didn't limit the maximum size of that queue, right?

Jieshan.

From: Cliff [cliffcheng...@gmail.com]
Sent: Saturday, November 22, 2014 12:32 PM
To: hbase-...@hadoop.apache.org
Subject: Questions about HBase replication

1.
Why does HBase replication need replicationSink?
I think replicationSource can do replicationSink's work as well.
And if we don't use replicationSink, we just need one time I/O.

2.
The queue added HLog path in replicationSource is PriorityBlockingQueue.
If the queue is full, HLog path cannot be added to the queue.
How to deal with the situation?




--
View this message in context: 
http://apache-hbase.679495.n3.nabble.com/Questions-about-HBase-replication-tp4066266.html
Sent from the HBase Developer mailing list archive at Nabble.com.

WritableRpcEngine$Invoker caches UGI

[Bijieshan]

Hi,

I think WritableRpcEngine$Invoker should not cache UGI. The intent behind this 
design?

We encounter 1 problem due to client called User#login more than 1 times. The 
error log:

14/05/08 15:44:40 FATAL ipc.SecureClient: SASL authentication failed. The most 
likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: 
No valid credentials provided (Mechanism level: Failed to find any Kerberos 
tgt)]
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at 
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:138)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:184)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$4(SecureClient.java:180)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:302)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:1)
at java.security.AccessController.doPrivileged(Native Method)

The reason of this exception as below:
(1). Client calls login.
(2). Calls Scan. So client side caches the proxy instance for HRegionInterface, 
and also caches the UGI.
(3). Client calls login again(It should not happen, but none protections we 
have).
(4). Wait until the TGT expire.
(5). Calls Scan again. So it will use the cached old UGI to create the 
SecureConnection.
(6). Gets a SaslException due to the invalid credential. So triggers 
SecureClient# handleSaslConnectionFailure
(7). Checks whether need to re-login by SecureClient# shouldAuthenticateOverKrb:
private synchronized boolean shouldAuthenticateOverKrb() throws IOException 
{
  UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
  UserGroupInformation currentUser = UserGroupInformation.getCurrentUser();
  UserGroupInformation realUser = currentUser.getRealUser();
  return relogin = authMethod == AuthMethod.KERBEROS  loginUser != null
  
  // Make sure user logged in using Kerberos either keytab or TGT
  loginUser.hasKerberosCredentials() 
  // relogin only in case it is the login user (e.g. JT)
  // or superuser (like oozie).
  (loginUser.equals(currentUser) || loginUser.equals(realUser));
}
The currentUser is the cached old one, but the login user is the new one 
created by step (3). So this check returns false and causes the re-login 
skipped.

The solution of this problem includes:

(1). Provide suggestion that the client side should only login once.
(2). Modify the logic in WritableRpcEngine$Invoker that does not cache UGI.  
Does this make sense?

Regards,
Jieshan.

RE: WritableRpcEngine$Invoker caches UGI

[Bijieshan]

Sent again since I got no reply.

That's 1 problem we encountered due to calling User#login more than 1 times. 
For your reference.

Jieshan.
-Original Message-
From: Bijieshan 
Sent: Thursday, May 08, 2014 6:54 PM
To: dev@hbase.apache.org
Cc: Fanghao
Subject: WritableRpcEngine$Invoker caches UGI

Hi,

I think WritableRpcEngine$Invoker should not cache UGI. The intent behind this 
design?

We encounter 1 problem due to client called User#login more than 1 times. The 
error log:

14/05/08 15:44:40 FATAL ipc.SecureClient: SASL authentication failed. The most 
likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: 
No valid credentials provided (Mechanism level: Failed to find any Kerberos 
tgt)]
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at 
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:138)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:184)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$4(SecureClient.java:180)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:302)
at 
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:1)
at java.security.AccessController.doPrivileged(Native Method)

The reason of this exception as below:
(1). Client calls login.
(2). Calls Scan. So client side caches the proxy instance for HRegionInterface, 
and also caches the UGI.
(3). Client calls login again(It should not happen, but none protections we 
have).
(4). Wait until the TGT expire.
(5). Calls Scan again. So it will use the cached old UGI to create the 
SecureConnection.
(6). Gets a SaslException due to the invalid credential. So triggers 
SecureClient# handleSaslConnectionFailure (7). Checks whether need to re-login 
by SecureClient# shouldAuthenticateOverKrb:
private synchronized boolean shouldAuthenticateOverKrb() throws IOException 
{
  UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
  UserGroupInformation currentUser = UserGroupInformation.getCurrentUser();
  UserGroupInformation realUser = currentUser.getRealUser();
  return relogin = authMethod == AuthMethod.KERBEROS  loginUser != null
  
  // Make sure user logged in using Kerberos either keytab or TGT
  loginUser.hasKerberosCredentials() 
  // relogin only in case it is the login user (e.g. JT)
  // or superuser (like oozie).
  (loginUser.equals(currentUser) || loginUser.equals(realUser));
}
The currentUser is the cached old one, but the login user is the new one 
created by step (3). So this check returns false and causes the re-login 
skipped.

The solution of this problem includes:

(1). Provide suggestion that the client side should only login once.
(2). Modify the logic in WritableRpcEngine$Invoker that does not cache UGI.  
Does this make sense?

Regards,
Jieshan.

RE: Please welcome our newest committer, Liang Xie!

[Bijieshan]

Congratulations, Liang Xie!

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: Thursday, December 19, 2013 8:47 AM
To: HBase Dev List
Subject: Please welcome our newest committer, Liang Xie!

谢良 has been doing great work for ages now; we're glad to have him on board 
(One of us!).  Keep up the great work 谢良.

St.Ack

RE: replication scopes in WAL

[Bijieshan]

Yes, it is only used for indicating which KVs to replicate. See 
ReplicationSource#removeNonReplicableEdits.

bq.If they indicate which KVs to replicate, wouldn't it be better to store 
bitmask or something like that?
Sounds like a good idea.

Regards,
Jieshan
-Original Message-
From: Sergey Shelukhin [mailto:ser...@hortonworks.com] 
Sent: Tuesday, April 16, 2013 9:30 AM
To: dev@hbase.apache.org
Subject: replication scopes in WAL

Hi. I am looking at converting WAL to PB and I see replication scopes in
WAL record.
Is there any documentation on those? The usage from code is not obvious,
there seems to be some usage implied that is not there.
If they indicate which KVs to replicate, wouldn't it be better to store
bitmask or something like that?

RE: Shared ThreadPoolExecutor in HTable by default.

[Bijieshan]

Use shared ThreadPoolExecutor will not always good, e.g. when we do metaScan or 
something else, we don't expect any delay due to no available threads in this 
shared pool.
But that would always be a good practice for application-level.
It is individual opinion only:)

Jieshan
-Original Message-
From: elliott.neil.cl...@gmail.com [mailto:elliott.neil.cl...@gmail.com] On 
Behalf Of Elliott Clark
Sent: Wednesday, April 03, 2013 6:45 AM
To: dev@hbase.apache.org
Subject: Shared ThreadPoolExecutor in HTable by default.

Is there any reason that the ThreadPoolExecutor used in HTable[1] couldn't
be a singleton. That would mean that by default htables share threads (as I
would argue is correct), but still give advanced users the ability to
override this using the  more explicit[2] constructor.


   1.
   
https://github.com/apache/hbase/blob/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L183
   2.
   
https://github.com/apache/hbase/blob/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L202

Thanks

RE: DISCUSSION: Component Lieutenants?

[Bijieshan]

I would like to volunteer for client, Scan, Filter, Region Splitting, Create 
Table(Table related operations)...
Thank you:)

Jieshan
-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: Wednesday, September 19, 2012 6:15 AM
To: dev@hbase.apache.org
Subject: Re: DISCUSSION: Component Lieutenants?

On Tue, Sep 18, 2012 at 12:37 PM, Gregory Chanan gcha...@cloudera.com wrote:
 How are we deciding what counts as a component?  Based on what people say
 here?

Yes.

I think that is ok.  If the thing can be described and someone has
volunteered to look after it, then it 'exists'.

 Some of these seem vastly different in scope (e.g. Client vs
 HalfStoreFile).


Agreed.

 Also, will it be obvious, from the JIRA, who I need to get reviews from and
 how many?  From Stack's e-mail it sounds like clicking on the component
 will give you a list of names; perhaps we should make it explicit in that
 link that one +1 is enough for anyone on this list, otherwise two +1s are
 necessary.

Good suggestion.   I'll do this.


 We need to make it clear what the process is for new
 contributors (more process is okay, but it needs to be fair and explicit).


Agreed.

I'll add a section to our community chapter in refguide w/ the few
rules we came up w/ here.

 What about patches that touch more than one component?


Both component owners?  Or a component owner and a +1 from someone
other than author (That'd align w/ our two +1s from any contributor
equating to a component owners' +1... sort of).

St.Ack

RE: Please welcome our newest committer and PMC member, Liyin Tang

[Bijieshan]

Congratulations Liyin!

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: Thursday, May 17, 2012 12:04 PM
To: HBase Dev List
Subject: Please welcome our newest committer and PMC member, Liyin Tang

One of us Liyin!

Keep up the great work.

Add yourself to the pom.xml team section (Don't break the build!).

St.Ack

RE: HBase Between Filters

[Bijieshan]

I think BinaryComparator will not give any help to achieve that goal(Because 
it's not a number comparison). 
You can try to customize your own comparator(Extend the class of 
WritableByteArrayComparable), and write the rules of how to do that comparison. 
And then using this comparator to do the scanning.

Jieshan

-Original Message-
From: Yifeng Jiang [mailto:uprushwo...@gmail.com] 
Sent: Monday, May 07, 2012 7:37 PM
To: dev@hbase.apache.org
Subject: Re: HBase Between Filters

Can you try adding this to your code:
scan.addColumn(Bytes.toBytes(TRAN), Bytes.toBytes(TRAN_ID))

-Yifeng

On May 4, 2012, at 10:04 PM, sanky999 wrote:

 I'm trying to retrieve rows with in range, using Filter List but I'm not
 successful. Below is my code snippet.
 
 I want to retrieve data between 1000 and 2000.
 
 HTable table = new HTable(conf, TRAN_DATA);
 
ListFilter filters = new ArrayListFilter();
 
SingleColumnValueFilter filter1 = new
 SingleColumnValueFilter(Bytes.toBytes(TRAN),
  Bytes.toBytes(TRAN_ID),
  CompareFilter.CompareOp.GREATER, new
 BinaryComparator(Bytes.toBytes(1000)));
filter1.setFilterIfMissing(true);
filters.add(filter1);
 
SingleColumnValueFilter filter2 = new
 SingleColumnValueFilter(Bytes.toBytes(TRAN),
  Bytes.toBytes(TRAN_ID),
  CompareFilter.CompareOp.LESS,new
 BinaryComparator(Bytes.toBytes(2000)));
 
filters.add(filter2);
 
FilterList filterList = new FilterList(filters);
 
Scan scan = new Scan();
scan.setFilter(filterList);
ResultScanner scanner1 = table.getScanner(scan);
 
System.out.println(Results of scan #1 - MUST_PASS_ALL:);
int n = 0;
 
for (Result result : scanner1) {
for (KeyValue kv : result.raw()) {
System.out.println(KV:  + kv + , Value: 
+ Bytes.toString(kv.getValue()));
{
n++;
 
}
}
scanner1.close();
 
 
 
 Tried with all possible ways using
 1. SingleColumnValueFilter filter2 = new
 SingleColumnValueFilter(Bytes.toBytes(TRANSACTIONS),
 Bytes.toBytes(TRANS_ID), CompareFilter.CompareOp.LESS, new
 SubstringComparator(5000));
 
 SingleColumnValueFilter filter2 = new
 SingleColumnValueFilter(Bytes.toBytes(TRANSACTIONS),
 Bytes.toBytes(TRANS_ID), CompareFilter.CompareOp.LESS,
 Bytes.toBytes(5000)); None of above approaches work :(
 
 --
 View this message in context: 
 http://apache-hbase.679495.n3.nabble.com/HBase-Between-Filters-tp3962242.html
 Sent from the HBase - Developer mailing list archive at Nabble.com.

Closed parent region present in Hlog.lastSeqWritten

[bijieshan]

Hi all,
We found so many hlogs in our cluster, after some analysis, we also found one 
splitted region occurred in HLog.lastSeqWritten. For this region had been 
closed, it can't be flushed again. So blocking all the other logs removing to 
.oldlogs directory.

05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many 
hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s): 
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed to 
schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many 
hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s): 
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed to 
schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many 
hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s): 
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:14:46,075 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed to 
schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
05:15:41,584 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many 
hlogs: logs=125, maxlogs=32; forcing flush of 1 regions(s): 
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:15:41,584 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed to 
schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null

Let's see what happened to the region of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r:

00:30:49,242 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished 
memstore flush of ~129.5m for region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
13299ms, sequenceid=20311822, compaction requested=true
00:30:49,242 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: 
Compaction requested for 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. because 
User-triggered split; priority=1, compaction queue size=5840
00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming flushed 
file at 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815
 to 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming flushed 
file at 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815
 to 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
00:30:59,614 INFO org.apache.hadoop.hbase.regionserver.Store: Added 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
 entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished 
memstore flush of ~133.5m for region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
21816ms, sequenceid=20312223, compaction requested=true
00:30:59,787 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: 
Compaction requested for 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. because 
regionserver20020.cacheFlusher; priority=0, compaction queue size=5840
00:31:12,605 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting 
compaction on region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.HRegion: completed 
compaction on region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. after 0sec
00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: 
Starting split of region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.: 
disabling compactions  flushes
00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates 
disabled for region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:13,718 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:39,552 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Offlined parent 
region Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
META
--
00:31:42,529 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/52ff3c7c6df6e0337876bbca29cee84a/value/973789709483406123.2acaf8e3acfd2e8a5825a1f6f0aca4a8,
 isReference=true, isBulkLoadResult=false, seqid=20312224, majorCompaction=false
00:31:42,532 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 

Re: Closed parent region present in Hlog.lastSeqWritten

[bijieshan]

Not including HBASE-5225(before 0.90.6.rc0). I think no relationship with this.
Thanks.
Jieshan

-邮件原件-
发件人: Ted Yu [mailto:yuzhih...@gmail.com] 
发送时间: 2012年1月25日 13:18
收件人: dev@hbase.apache.org
抄送: u...@hbase.apache.org; Chenjian
主题: Re: Closed parent region present in Hlog.lastSeqWritten

Can you provide more detail on this 0.90.5 +, please ?

Did it include HBASE-5225 ?

Thanks

On Tue, Jan 24, 2012 at 8:55 PM, bijieshan bijies...@huawei.com wrote:

 Hi all,
 We found so many hlogs in our cluster, after some analysis, we also found
 one splitted region occurred in HLog.lastSeqWritten. For this region had
 been closed, it can't be flushed again. So blocking all the other logs
 removing to .oldlogs directory.

 05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
 hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s):
 2acaf8e3acfd2e8a5825a1f6f0aca4a8
 05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
 to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
 05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
 hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s):
 2acaf8e3acfd2e8a5825a1f6f0aca4a8
 05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
 to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
 05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
 hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s):
 2acaf8e3acfd2e8a5825a1f6f0aca4a8
 05:14:46,075 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
 to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null
 05:15:41,584 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
 hlogs: logs=125, maxlogs=32; forcing flush of 1 regions(s):
 2acaf8e3acfd2e8a5825a1f6f0aca4a8
 05:15:41,584 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
 to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null, requester=null

 Let's see what happened to the region of
 2acaf8e3acfd2e8a5825a1f6f0aca4a8r:

 00:30:49,242 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished
 memstore flush of ~129.5m for region
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in
 13299ms, sequenceid=20311822, compaction requested=true
 00:30:49,242 DEBUG
 org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction
 requested for
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 because User-triggered split; priority=1, compaction queue size=5840
 00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming
 flushed file at hdfs://
 192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815to
  hdfs://
 192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
 00:30:55,214http://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123%0A00:30:55,214INFO
  org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at
 hdfs://
 192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815to
  hdfs://
 192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
 00:30:59,614http://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123%0A00:30:59,614INFO
  org.apache.hadoop.hbase.regionserver.Store: Added hdfs://
 192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
 entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
 00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished
 memstore flush of ~133.5m for region
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in
 21816ms, sequenceid=20312223, compaction requested=true
 00:30:59,787 DEBUG
 org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction
 requested for
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 because regionserver20020.cacheFlusher; priority=0, compaction queue
 size=5840
 00:31:12,605 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting
 compaction on region
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.HRegion: completed
 compaction on region
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. after
 0sec
 00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction:
 Starting split of region
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.:
 disabling compactions  flushes
 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates
 disabled for region

Re: Closed parent region present in Hlog.lastSeqWritten

[bijieshan]

Ted:

Below is the search info for 20312224 20312223 20312225. We also analyzed 
this but got nothing:(

[20312224]
00:31:42,529 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/52ff3c7c6df6e0337876bbca29cee84a/value/973789709483406123.2acaf8e3acfd2e8a5825a1f6f0aca4a8,
 isReference=true, isBulkLoadResult=false, seqid=20312224, majorCompaction=false
00:31:42,541 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined 
Htable_UFDR_031,00332,1325809872607.259d0c620c9105928e52713f4a5a252e.; next 
sequenceid=20312224
00:31:42,595 INFO org.apache.hadoop.hbase.regionserver.Store: Started 
compaction of 10 file(s) in cf=value, hasReferences=true, into 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/52ff3c7c6df6e0337876bbca29cee84a/.tmp,
 seqid=20312224, totalSize=1.6g
00:34:48,061 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Found 1 hlogs 
to remove out of total 4; oldest outstanding sequenceid is 20312224 from region 
2acaf8e3acfd2e8a5825a1f6f0aca4a8
01:39:14,494 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/52ff3c7c6df6e0337876bbca29cee84a/value/1634833527842694834,
 isReference=false, isBulkLoadResult=false, seqid=20312224, 
majorCompaction=false
01:55:15,293 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/52ff3c7c6df6e0337876bbca29cee84a/value/1634833527842694834,
 isReference=false, isBulkLoadResult=false, seqid=20312224, 
majorCompaction=false

[20312223]
00:30:59,614 INFO org.apache.hadoop.hbase.regionserver.Store: Added 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
 entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished 
memstore flush of ~133.5m for region 
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
21816ms, sequenceid=20312223, compaction requested=true
00:31:42,532 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded 
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/259d0c620c9105928e52713f4a5a252e/value/973789709483406123.2acaf8e3acfd2e8a5825a1f6f0aca4a8,
 isReference=true, isBulkLoadResult=false, seqid=20312223, majorCompaction=false

[20312225]  
00:31:42,531 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined 
Htable_UFDR_031,003732800093168-03594291912,1325809872607.52ff3c7c6df6e0337876bbca29cee84a.;
 next sequenceid=20312225   


-邮件原件-
发件人: Ted Yu [mailto:yuzhih...@gmail.com] 
发送时间: 2012年1月25日 14:33
收件人: dev@hbase.apache.org
抄送: u...@hbase.apache.org; Chenjian
主题: Re: Closed parent region present in Hlog.lastSeqWritten

Jieshan:
Can you search the region server log for seq id: 20312224 ?

Thanks

2012/1/24 Ramkrishna.S.Vasudevan ramkrishna.vasude...@huawei.com

 HBASE-5225 was merged for different purpose.  But this issue does not seem
 to be because of HBASE-5225.
 Because in HBASE-5225 we tried to fix the case where the seq id is missed.
 Here it is not missed it is still there in lastSeqWritten without getting
 flushed.

 Regards
 Ram

 -Original Message-
 From: bijieshan [mailto:bijies...@huawei.com]
 Sent: Wednesday, January 25, 2012 10:59 AM
 To: u...@hbase.apache.org; dev@hbase.apache.org
 Cc: Chenjian
 Subject: Re: Closed parent region present in Hlog.lastSeqWritten

 Not including HBASE-5225(before 0.90.6.rc0). I think no relationship with
 this.
 Thanks.
 Jieshan

 -邮件原件-
 发件人: Ted Yu [mailto:yuzhih...@gmail.com]
 发送时间: 2012年1月25日 13:18
 收件人: dev@hbase.apache.org
 抄送: u...@hbase.apache.org; Chenjian
 主题: Re: Closed parent region present in Hlog.lastSeqWritten

 Can you provide more detail on this 0.90.5 +, please ?

 Did it include HBASE-5225 ?

 Thanks

 On Tue, Jan 24, 2012 at 8:55 PM, bijieshan bijies...@huawei.com wrote:

  Hi all,
  We found so many hlogs in our cluster, after some analysis, we also found
  one splitted region occurred in HLog.lastSeqWritten. For this region had
  been closed, it can't be flushed again. So blocking all the other logs
  removing to .oldlogs directory.
 
  05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 requester=null
  05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 requester=null
  05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s

A concurrency issue on SoftValueSortedMap?

[bijieshan]

Hi all,

We use thousand of threads doing the scan operations. One thread got blocked at 
the position of java.util.TreeMap.fixAfterDeletion. It can't come out of the 
loop of TreeMap.fixAfterDeletion:

So I think it maybe a concurrency issue. Has someone encountered this?
Thank you.

Jieshan.

Here's the thread dump:

Thread-923 prio=10 tid=0x7f3d40553000 nid=0x3ed6 runnable 
[0x7f3d05c1b000]
   java.lang.Thread.State: RUNNABLE
at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2176)
at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
at java.util.TreeMap.remove(TreeMap.java:585)
at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
at 
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
- locked 0x7f3d92937748 (a 
org.apache.hadoop.hbase.util.SoftValueSortedMap)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at 
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at 
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at 
org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at 
com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
at 
com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
at framework.QueryThread.run(QueryThread.java:47)

Thread-923 prio=10 tid=0x7f3d40553000 nid=0x3ed6 runnable 
[0x7f3d05c1a000]
   java.lang.Thread.State: RUNNABLE
at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2193)
at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
at java.util.TreeMap.remove(TreeMap.java:585)
at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
at 
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
- locked 0x7f3d94f24f70 (a 
org.apache.hadoop.hbase.util.SoftValueSortedMap)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at 
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at 
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at 
org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at 
com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
at 
com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
at framework.QueryThread.run(QueryThread.java:47)

Re: A concurrency issue on SoftValueSortedMap?

[bijieshan]

Sorry, something important I missed in my last email.
SoftValueSortedMap uses TreeMap. It has some synchronized methods to edit the 
data from this map. If we use these methods, it’s ok. But in 
HConnectionManager#getCachedLocation:

  // Cut the cache so that we only get the part that could contain
  // regions that match our key
  SoftValueSortedMapbyte[], HRegionLocation matchingRegions =
tableLocations.headMap(row);

  // if that portion of the map is empty, then we're done. otherwise,
  // we need to examine the cached location to verify that it is
  // a match by end key as well.
  if (!matchingRegions.isEmpty()) {
HRegionLocation possibleRegion = null;
try {
  possibleRegion = matchingRegions.get(matchingRegions.lastKey());
} catch (NoSuchElementException nsee) {
  LOG.warn(checkReferences() might have removed the key, nsee);
}
SoftValueSortedMap#get:
  public synchronized V get(Object key) {

  checkReferences();
SoftValueK,V value = this.internalMap.get(key);
if (value == null) {
  return null;
}
if (value.get() == null) {
  this.internalMap.remove(key);
  return null;
}
return value.get();
  }

In this method, it can remove the element from internalMap which is a heapMap 
from the backed TreeMap. A concurrency issue may happen from here.
Correct me if am wrong.
Thank you.

Jieshan.

发件人: bijieshan
发送时间: 2011年12月21日 18:48
收件人: dev@hbase.apache.org; u...@hbase.apache.org
抄送: wenzaohua; Chenjian
主题: A concurrency issue on SoftValueSortedMap?

Hi all,

We use thousand of threads doing the scan operations. One thread got blocked at 
the position of “java.util.TreeMap.fixAfterDeletion”. It can’t come out of the 
loop of “TreeMap.fixAfterDeletion”:

So I think it maybe a concurrency issue. Has someone encountered this?
Thank you.

Jieshan.

Here’s the thread dump:

Thread-923 prio=10 tid=0x7f3d40553000 nid=0x3ed6 runnable 
[0x7f3d05c1b000]
   java.lang.Thread.State: RUNNABLE
at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2176)
at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
at java.util.TreeMap.remove(TreeMap.java:585)
at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
at 
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
- locked 0x7f3d92937748 (a 
org.apache.hadoop.hbase.util.SoftValueSortedMap)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at 
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at 
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at 
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at 
org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at 
com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
at 
com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
at framework.QueryThread.run(QueryThread.java:47)

Thread-923 prio=10 tid=0x7f3d40553000 nid=0x3ed6 runnable 
[0x7f3d05c1a000]
   java.lang.Thread.State: RUNNABLE
at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2193)
at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
at java.util.TreeMap.remove(TreeMap.java:585)
at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
at 
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
- locked 0x7f3d94f24f70 (a 
org.apache.hadoop.hbase.util.SoftValueSortedMap)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
at 
org.apache.hadoop.hbase.client.HConnectionManager

Re: Coprocessors dynamic jar loading and coprocessorExec

[bijieshan]

In my understanding, all the coprocessors are loaded in the initialization of 
coprocessor environment, dynamic loading is not supported currently. 
Maybe we can add this new feature.

Jieshan

--
发件人: Andrei Dragomir [mailto:adrag...@adobe.com] 
发送时间: 2011年12月2日 17:07
收件人: dev@hbase.apache.org
主题: Coprocessors dynamic jar loading and coprocessorExec

I'm running into a bit of an issue with table coprocessors loaded
dynamically. 

I'm setting a table coprocessor from the shell (dynamic loading),
everything works fine. The coprocessor is not a RegionObserver /
WALObserver, it just has a custom method.

I see in the logs the coprocessor class being loaded correctly, it is
displayed in the web interface, everything seems to be fine.

When I try to call the custom method, the call throws an error. I tracked
it down to o.a.h.h.client.coprocessor.Exec, which tries to eagerly
deserialize the call. The problem is that in the context where the Exec
call is running, there is no coprocessor protocol class loaded (it's
dynamic, not from the configuration), so the call to get the class fails:

protocol = (ClassCoprocessorProtocol)conf.getClassByName(protocolName);

I'm just trying to understand whether this is by design, or if it's a
missing feature. 
I have a patch that passes the coprocessor class as String, and the actual
instantiation is done in the HRegion instance, where we actually have the
coprocessor class. I can create an issue and attach it.

Thanks, 
  Andrei. 

Re: Suspected memory leak

[bijieshan]

Thank you all. 
I think it's the same problem with the link provided by Stack. Because the 
heap-size is stabilized, but the non-heap size keep growing. So I think not the 
problem of the CMS GC bug. 
And we have known the content of the problem memory section, all the records 
contains the info like below:
|www.hostname02087075.comlhggmdjapwpfvkqvxgnskzzydiywoacjnpljkarlehrnzzbpbxc||460|||Agent
BBZHtable_UFDR_058,048342220093168-02570


Jieshan.

-邮件原件-
发件人: Kihwal Lee [mailto:kih...@yahoo-inc.com] 
发送时间: 2011年12月2日 4:20
收件人: dev@hbase.apache.org
抄送: Ramakrishna s vasudevan; u...@hbase.apache.org
主题: Re: Suspected memory leak

Adding to the excellent write-up by Jonathan:
Since finalizer is involved, it takes two GC cycles to collect them.  Due to a 
bug/bugs in the CMS GC, collection may not happen and the heap can grow really 
big.  See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7112034 for 
details.

Koji tried -XX:-CMSConcurrentMTEnabled and confirmed that all the socket 
related objects were being collected properly. This option forces the 
concurrent marker to be one thread. This was for HDFS, but I think the same 
applies here.

Kihwal

On 12/1/11 1:26 PM, Stack st...@duboce.net wrote:

Make sure its not the issue that Jonathan Payne identifiied a while
back: 
https://groups.google.com/group/asynchbase/browse_thread/thread/c45bc7ba788b2357#
St.Ack

Re: Round Robin Assignment of enabling table HBASE-4669 - Was: HBASE-4669- Needed for 0.90.x version

[bijieshan]

I also hope this patch can be integrated into 0.90.
Thank you for your efforts on this issue:)

Jieshan.

-邮件原件-
发件人: Ted Yu [mailto:yuzhih...@gmail.com] 
发送时间: 2011年11月3日 22:34
收件人: dev@hbase.apache.org; ramakrish...@huawei.com
主题: Re: Round Robin Assignment of enabling table HBASE-4669 - Was: HBASE-4669- 
Needed for 0.90.x version

+1 on backporting.

Cheers

On Thu, Nov 3, 2011 at 5:00 AM, Ramkrishna S Vasudevan 
ramakrish...@huawei.com wrote:


 Hi All

 HBASE-4669 (Bijieshan's patch) is an useful feature and can be good even in
 0.90.x.  Though it is an improvement i feel it will be a good value add .
 Infact as part of our application we need a feature of that sort.
 Also i think it will not be a problem to the existing feature.  Pls provide
 your inputs and suggestions if it is ok to backport HBASE-4669 to 0.90.x.

 Regards
 Ram




  _

Suggest to add a choice of using round-robin assignment on enable-table

[bijieshan]

Hi,

Under some scenarios, we use the function of disable/enable HTable. But 
currently, enable HTable using the random-assignment. We hope all the regions 
show a better distribution, no matter how many regions and how many 
regionservers.

So I suggest to add a choice of using round-robin assignment on enable-table.

Does this seem reasonable?

Thanks,
Jieshan

Re: Please welcome Ramkrishna S. Vasudevan, our newest hbase committer

[bijieshan]

Congratulations! Ram:)

-邮件原件-
发件人: Ramakrishna S Vasudevan 00902313 [mailto:ramakrish...@huawei.com] 
发送时间: 2011年9月29日 2:36
收件人: dev@hbase.apache.org
抄送: dev@hbase.apache.org
主题: Re: Please welcome Ramkrishna S. Vasudevan, our newest hbase committer


Dear All

Thanks a lot.

Hope to do my best with all your guys support. :)

Thanks once again.

Regards
Ram


- Original Message -
From: Doug Meil doug.m...@explorysmedical.com
Date: Wednesday, September 28, 2011 11:55 pm
Subject: Re: Please welcome Ramkrishna S. Vasudevan, our newest hbase committer
To: dev@hbase.apache.org dev@hbase.apache.org

 
 Welcome!
 
 
 
 
 
 On 9/28/11 12:15 PM, Stack st...@duboce.net wrote:
 
 Please welcome Ramkrishna, our newest hbase committer.  Ram has been
 going great guns fixing ugly hbase bugs with a good while now.  I'm
 glad he's on board.
 
 Good on you Ram,
 St.Ack
 
 

About why did the unit tests get killed//Re: Build failed in Jenkins: hbase-0.90 #282

[bijieshan]

About this failure, I've seen it several times. I didn't see the detail logs 
for this one. But in my env, there's always some similar logs :

2011-09-02 09:33:57,517 INFO  [RS_CLOSE_META-linux1.site,53487,1314927226948-0] 
regionserver.HRegion(554): Closed .META.,,1.1028785192
2011-09-02 09:33:57,517 DEBUG [RS_CLOSE_META-linux1.site,53487,1314927226948-0] 
handler.CloseRegionHandler(138): Closed region .META.,,1.1028785192
2011-09-02 09:33:57,564 INFO  [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
regionserver.Store(494): Renaming flushed file at 
hdfs://localhost:47319/user/root/-ROOT-/70236052/.tmp/1398417344723896979 to 
hdfs://localhost:47319/user/root/-ROOT-/70236052/info/3663494863054324738
2011-09-02 09:33:57,582 INFO  [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
regionserver.Store(504): Added 
hdfs://localhost:47319/user/root/-ROOT-/70236052/info/3663494863054324738, 
entries=2, sequenceid=40, memsize=464.0, filesize=504.0
2011-09-02 09:33:57,582 INFO  [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
regionserver.HRegion(1026): Finished memstore flush of ~464.0 for region 
-ROOT-,,0.70236052 in 847ms, sequenceid=40, compaction requested=false
2011-09-02 09:33:57,583 DEBUG [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
regionserver.Store(418): closed info
2011-09-02 09:33:57,583 INFO  [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
regionserver.HRegion(554): Closed -ROOT-,,0.70236052
2011-09-02 09:33:57,583 DEBUG [RS_CLOSE_ROOT-linux1.site,53487,1314927226948-0] 
handler.CloseRegionHandler(138): Closed region -ROOT-,,0.70236052
2011-09-02 09:33:57,700 INFO  
[RegionServer:0;linux1.site,53487,1314927226948.leaseChecker] 
regionserver.Leases(124): 
RegionServer:0;linux1.site,53487,1314927226948.leaseChecker closing leases
2011-09-02 09:33:57,701 INFO  
[RegionServer:0;linux1.site,53487,1314927226948.leaseChecker] 
regionserver.Leases(131): 
RegionServer:0;linux1.site,53487,1314927226948.leaseChecker closed leases
2011-09-02 09:33:58,230 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:33:58,460 INFO  [RegionServer:0;linux1.site,53487,1314927226948] 
regionserver.HRegionServer(709): Waiting on 1 regions to close
2011-09-02 09:33:58,460 DEBUG [RegionServer:0;linux1.site,53487,1314927226948] 
regionserver.HRegionServer(713): 
{5a8aaf25ba0a3afd05130cb9ccda5b47=test,ddd,1314927235514.5a8aaf25ba0a3afd05130cb9ccda5b47.}
2011-09-02 09:33:59,230 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:00,230 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:01,231 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:02,231 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:03,231 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:04,232 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
2011-09-02 09:34:05,232 INFO  [Master:0;linux1.site:40897] 
master.ServerManager(465): Waiting on regionserver(s) to go down 
linux1.site,53487,1314927226948
 
I've taken a detailed analysis. I think it happens in the below scenario:

One region is at the end of the opening process. But during the time, the 
cluster is shutting down. The user regions will be closed, and then .META., 
-ROOT-.
Step A: Cluster shut down.
Step B: Close user regions.
Step C: Close .META. and -ROOT-.

The new region's opening started before the step A, but finished after step B 
but before step C. So it will be added into onlineRegions. 
Then the .META. and -ROOT- get closed. But the new region can't be closed 
anymore. 

It's a race here.

Please correct me if am wrong.

Jieshan

---
发件人: Apache Jenkins Server [mailto:jenk...@builds.apache.org] 
发送时间: 2011年8月31日 18:23
收件人: dev@hbase.apache.org
主题: Build failed in Jenkins: hbase-0.90 #282

See https://builds.apache.org/job/hbase-0.90/282/changes

Changes:

[tedyu] revert HBASE-4269 to see if 'Not a host:port pair' error still exists

--
[...truncated 1088 lines...]
Running org.apache.hadoop.hbase.regionserver.TestColumnSeeking
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.961 sec
Running org.apache.hadoop.hbase.client.TestMultipleTimestamps
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 73.359 sec
Running org.apache.hadoop.hbase.regionserver.TestMemStoreLAB
Tests run: 3, Failures: 0, Errors: 0,