[jira] [Created] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager

[stack (JIRA)]

stack created HBASE-17844:
-

 Summary: Subset of HBASE-14614, Procedure v2: Core Assignment 
Manager
 Key: HBASE-17844
 URL: https://issues.apache.org/jira/browse/HBASE-17844
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0
Reporter: stack
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/551/artifact/website.patch.zip
 | funzip > f159557eded160680e623b966350ea3442b5f35a.patch
  git fetch
  git checkout -b asf-site-f159557eded160680e623b966350ea3442b5f35a 
origin/asf-site
  git am --whitespace=fix f159557eded160680e623b966350ea3442b5f35a.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-f159557eded160680e623b966350ea3442b5f35a branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-f159557eded160680e623b966350ea3442b5f35a:asf-site
  git checkout asf-site
  git branch -D asf-site-f159557eded160680e623b966350ea3442b5f35a

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/551/console

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Yu Li]

+1 on -alpha/-beta, and cannot wait to see an alpha1 out (smile)

Best Regards,
Yu

On 29 March 2017 at 10:28, 张铎(Duo Zhang)  wrote:

> +1 on 2.0.0-alpha[x]/2.0.0-beta[x].
>
> 2017-03-29 10:07 GMT+08:00 Andrew Purtell :
>
> > That settles it. :-)
> >
> > I'd also be cool with -alpha, -beta, etc.
> >
> > > On Mar 28, 2017, at 1:25 PM, Enis Söztutar  wrote:
> > >
> > > I would automatically -1 any release with a number like 1.99 regardless
> > of
> > > content.
> > >
> > > Semantic versioning which we are following already provides an answer
> for
> > > this:
> > > http://semver.org/#spec-item-9
> > >
> > > From my experience as RM for 0.99.x series and 1.0.x series, I would
> > > suggest we do 2.0.0-alpha1 and alpha2, and one or two betas. I think we
> > > should start the alpha1 release now which does not have to wait for
> > > anything but packaging work.
> > >
> > > Enis
> > >
> > >> On Tue, Mar 28, 2017 at 1:19 PM, Sean Busbey 
> wrote:
> > >>
> > >> Hi folks!
> > >>
> > >> What are folks opinions on how we name releases leading up to HBase
> > >> 2.0 that aren't quite done yet?
> > >>
> > >> For 1.0, we used 0.99 as a placeholder for "what we expect will be in
> > >> 1.0 but is not yet ready for production use." That got us 0.99.0,
> > >> 0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
> > >> continuing this pattern would be done with 1.99, I suppose.
> > >>
> > >> This issue I take with this approach is that back before 1.0, we could
> > >> count on users thinking of 0.99 as a different major release train
> > >> than 0.98. Now, I'm concerned that some might lump 1.99 in with the
> > >> 1.y major release series.
> > >>
> > >> Alternatively we could expressly label the releases as alpha/beta
> > >> based on our confidence. This would give us 2.0.0-alpha1,
> > >> 2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
> > >> futzing with sort order, but clearly conveys that these releases are
> > >> both part of what will be the 2.y major release series and not for the
> > >> faint of heart.
> > >>
> > >>
> > >> Thoughts?
> > >>
> >
>

Re: How threads interact with each other in HBase

[杨苏立 Yang Su Li]

Hi Josh,

Thanks a lot for your response and we really appreciate the effort you put
in to help us.

Following are responses to your comments:

1. We understand there is a block cache, and processing an RPC may only
involve looking up the cache instead of going to HDFS. That's why we said
"If the processing of RPC requires reading data from HDFS...".

2. Thanks a lot for pointing out that we over-simplified the write path in
our description (though they are reflected in the graph). I have updated
the document ( http://pages.cs.wisc.edu/~suli/hbase.pdf ) so that the
description is more accurate. We wonder if you could take a look to see if
the description now agrees with the reality.
We understand that by default put only returns after the corresponding WAL
entry is persisted. When we say "asynchronous write", we are referring to
the writes issued by the HBase to HDFS. Though the write(append) to the WAL
is alway synchronous (there are dedicated LOG sync threads continuously
issue flush/sync to HDFS), other writes HBase does (e.g., the memstore
flush writes) can be asynchronous as there no file.flush() or file.sync()
following the writes.

Again, thanks a lot!

Suli

On Tue, Mar 28, 2017 at 12:11 PM, Josh Elser  wrote:

> 1.1 -> 2: don't forget about the block cache which can invalidate the need
> for any HDFS read.
>
> I think you're over-simplifying the write-path quite a bit. I'm not sure
> what you mean by an 'asynchronous write', but that doesn't exist at the
> HBase RPC layer as that would invalidate the consistency guarantees (if an
> RPC returns to the client that data was "put", then it is durable).
>
> Going off of memory (sorry in advance if I misstate something): the
> general way that data is written to the WAL is a "group commit". You have
> many threads all trying to append data to the WAL -- performance would be
> terrible if you serially applied all of these writes. Instead, many writes
> can be accepted and a the caller receives a Future. The caller must wait
> for the Future to complete. What's happening behind the scene is that the
> writes are being bundled together to reduce the number of syncs to the WAL
> ("grouping" the writes together). When one caller's future would complete,
> what really happened is that the write/sync which included the caller's
> update was committed (along with others). All of this is happening inside
> the RS's implementation of accepting an update.
>
> https://github.com/apache/hbase/blob/55d6dcaf877cc5223e67973
> 6eb613173229c18be/hbase-server/src/main/java/org/apache/hadoop/hbase/
> regionserver/wal/FSHLog.java#L74-L106
>
>
> 杨苏立 Yang Su Li wrote:
>
>> The attachment can be found in the following URL:
>> http://pages.cs.wisc.edu/~suli/hbase.pdf
>>
>> Sorry for the inconvenience...
>>
>>
>> On Mon, Mar 27, 2017 at 8:25 PM, Ted Yu  wrote:
>>
>> Again, attachment didn't come thru.
>>>
>>> Is it possible to formulate as google doc ?
>>>
>>> Thanks
>>>
>>> On Mon, Mar 27, 2017 at 6:19 PM, 杨苏立 Yang Su Li
>>> wrote:
>>>
>>> Hi,

 I am a graduate student working on scheduling on storage systems, and we
 are interested in how different threads in HBase interact with each
 other
 and how it might affect scheduling.

 I have written down my understanding on how HBase/HDFS works based on
 its
 current thread architecture (attached). I am wondering if the developers

>>> of
>>>
 HBase could take a look at it and let me know if anything is incorrect
 or
 inaccurate, or if I have missed anything.

 Thanks a lot for your help!

 On Wed, Mar 22, 2017 at 3:39 PM, 杨苏立 Yang Su Li
 wrote:

 Hi,
>
> I am a graduate student working on scheduling on storage systems, and
> we
> are interested in how different threads in HBase interact with each
>
 other
>>>
 and how it might affect scheduling.
>
> I have written down my understanding on how HBase/HDFS works based on
>
 its
>>>
 current thread architecture (attached). I am wondering if the
>
 developers of
>>>
 HBase could take a look at it and let me know if anything is incorrect
>
 or
>>>
 inaccurate, or if I have missed anything.
>
> Thanks a lot for your help!
>
> --
> Suli Yang
>
> Department of Physics
> University of Wisconsin Madison
>
> 4257 Chamberlin Hall
> Madison WI 53703
>
>
>
 --
 Suli Yang

 Department of Physics
 University of Wisconsin Madison

 4257 Chamberlin Hall
 Madison WI 53703



>>
>>
>>


-- 
Suli Yang

Department of Physics
University of Wisconsin Madison

4257 Chamberlin Hall
Madison WI 53703

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Duo Zhang]

+1 on 2.0.0-alpha[x]/2.0.0-beta[x].

2017-03-29 10:07 GMT+08:00 Andrew Purtell :

> That settles it. :-)
>
> I'd also be cool with -alpha, -beta, etc.
>
> > On Mar 28, 2017, at 1:25 PM, Enis Söztutar  wrote:
> >
> > I would automatically -1 any release with a number like 1.99 regardless
> of
> > content.
> >
> > Semantic versioning which we are following already provides an answer for
> > this:
> > http://semver.org/#spec-item-9
> >
> > From my experience as RM for 0.99.x series and 1.0.x series, I would
> > suggest we do 2.0.0-alpha1 and alpha2, and one or two betas. I think we
> > should start the alpha1 release now which does not have to wait for
> > anything but packaging work.
> >
> > Enis
> >
> >> On Tue, Mar 28, 2017 at 1:19 PM, Sean Busbey  wrote:
> >>
> >> Hi folks!
> >>
> >> What are folks opinions on how we name releases leading up to HBase
> >> 2.0 that aren't quite done yet?
> >>
> >> For 1.0, we used 0.99 as a placeholder for "what we expect will be in
> >> 1.0 but is not yet ready for production use." That got us 0.99.0,
> >> 0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
> >> continuing this pattern would be done with 1.99, I suppose.
> >>
> >> This issue I take with this approach is that back before 1.0, we could
> >> count on users thinking of 0.99 as a different major release train
> >> than 0.98. Now, I'm concerned that some might lump 1.99 in with the
> >> 1.y major release series.
> >>
> >> Alternatively we could expressly label the releases as alpha/beta
> >> based on our confidence. This would give us 2.0.0-alpha1,
> >> 2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
> >> futzing with sort order, but clearly conveys that these releases are
> >> both part of what will be the 2.y major release series and not for the
> >> faint of heart.
> >>
> >>
> >> Thoughts?
> >>
>

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Andrew Purtell]

That settles it. :-)

I'd also be cool with -alpha, -beta, etc. 

> On Mar 28, 2017, at 1:25 PM, Enis Söztutar  wrote:
> 
> I would automatically -1 any release with a number like 1.99 regardless of
> content.
> 
> Semantic versioning which we are following already provides an answer for
> this:
> http://semver.org/#spec-item-9
> 
> From my experience as RM for 0.99.x series and 1.0.x series, I would
> suggest we do 2.0.0-alpha1 and alpha2, and one or two betas. I think we
> should start the alpha1 release now which does not have to wait for
> anything but packaging work.
> 
> Enis
> 
>> On Tue, Mar 28, 2017 at 1:19 PM, Sean Busbey  wrote:
>> 
>> Hi folks!
>> 
>> What are folks opinions on how we name releases leading up to HBase
>> 2.0 that aren't quite done yet?
>> 
>> For 1.0, we used 0.99 as a placeholder for "what we expect will be in
>> 1.0 but is not yet ready for production use." That got us 0.99.0,
>> 0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
>> continuing this pattern would be done with 1.99, I suppose.
>> 
>> This issue I take with this approach is that back before 1.0, we could
>> count on users thinking of 0.99 as a different major release train
>> than 0.98. Now, I'm concerned that some might lump 1.99 in with the
>> 1.y major release series.
>> 
>> Alternatively we could expressly label the releases as alpha/beta
>> based on our confidence. This would give us 2.0.0-alpha1,
>> 2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
>> futzing with sort order, but clearly conveys that these releases are
>> both part of what will be the 2.y major release series and not for the
>> faint of heart.
>> 
>> 
>> Thoughts?
>> 

Re: [ANNOUNCE] - Welcome our new HBase committer Anastasia Braginsky

[宾莉金（binlijin）]

Congratulations!

2017-03-28 22:16 GMT+08:00 Kahlil Oppenheimer :

> Congrats Anastasia!
>
> On Tue, Mar 28, 2017 at 9:50 AM Nick Dimiduk  wrote:
>
> > Thank you for all your contributions Anastasia!
> >
> > On Mon, Mar 27, 2017 at 5:38 AM ramkrishna vasudevan <
> > ramkrishna.s.vasude...@gmail.com> wrote:
> >
> > > Hi All
> > >
> > > Welcome Anastasia Braginsky, one more female committer to HBase. She
> has
> > > been active now for a while with her Compacting memstore feature and
> she
> > > along with Eshcar have done lot of talks in various meetups and
> HBaseCon
> > on
> > > their feature.
> > >
> > > Welcome onboard and looking forward to work with you Anastasia !!!
> > >
> > > Regards
> > > Ram
> > >
> >
>



-- 
*Best Regards,*
 lijin bin

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/550/artifact/website.patch.zip
 | funzip > 0345fc87759a7d44ecc385327ebb586fc632fb65.patch
  git fetch
  git checkout -b asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65 
origin/asf-site
  git am --whitespace=fix 0345fc87759a7d44ecc385327ebb586fc632fb65.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65 branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65:asf-site
  git checkout asf-site
  git branch -D asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/550/console

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/549/artifact/website.patch.zip
 | funzip > 0345fc87759a7d44ecc385327ebb586fc632fb65.patch
  git fetch
  git checkout -b asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65 
origin/asf-site
  git am --whitespace=fix 0345fc87759a7d44ecc385327ebb586fc632fb65.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65 branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65:asf-site
  git checkout asf-site
  git branch -D asf-site-0345fc87759a7d44ecc385327ebb586fc632fb65

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/549/console

[jira] [Resolved] (HBASE-17727) [C++] Make RespConverter work with RawAsyncTableImpl

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar resolved HBASE-17727.
---
   Resolution: Fixed
Fix Version/s: HBASE-14850

> [C++] Make RespConverter work with RawAsyncTableImpl
> 
>
> Key: HBASE-17727
> URL: https://issues.apache.org/jira/browse/HBASE-17727
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Enis Soztutar
> Fix For: HBASE-14850
>
> Attachments: hbase-17727-v1.patch
>
>
> This is a follow up work of HBASE-17465. 
> There's a problem to dereference instance of RpcCallback when it's passed as 
> function argument.
> {code}
> template  typename S>
> using RespConverter = std::function;
> {code}
> {code}
>   templatetypename PREQ,
>typename PRESP,
>typename RESP>
>   folly::Future Call(
>   std::shared_ptr rpc_client,
>   std::shared_ptr controller,
>   std::shared_ptr loc,
>   const REQ& req,
>   const ReqConverter& 
> req_converter,
>   const hbase::RpcCall& rpc_call,
>   const RespConverter& resp_converter) {
> rpc_call(
> rpc_client,
> loc,
> controller,
> std::move(req_converter(req, loc->region_name(
> .then([&, this](std::unique_ptr presp) {
>   // std::unique_ptr result = resp_converter(presp);
>   std::unique_ptr result = 
> hbase::ResponseConverter::FromGetResponse(*presp);
>   promise_->setValue(std::move(*result));
> })
> .onError([this] (const std::exception& e) 
> {promise_->setException(e);});
> return promise_->getFuture();
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Ted Yu]

2.0.0-alpha / 2.0.0-beta sound good to me.

On Tue, Mar 28, 2017 at 1:25 PM, Enis Söztutar  wrote:

> I would automatically -1 any release with a number like 1.99 regardless of
> content.
>
> Semantic versioning which we are following already provides an answer for
> this:
> http://semver.org/#spec-item-9
>
> From my experience as RM for 0.99.x series and 1.0.x series, I would
> suggest we do 2.0.0-alpha1 and alpha2, and one or two betas. I think we
> should start the alpha1 release now which does not have to wait for
> anything but packaging work.
>
> Enis
>
> On Tue, Mar 28, 2017 at 1:19 PM, Sean Busbey  wrote:
>
> > Hi folks!
> >
> > What are folks opinions on how we name releases leading up to HBase
> > 2.0 that aren't quite done yet?
> >
> > For 1.0, we used 0.99 as a placeholder for "what we expect will be in
> > 1.0 but is not yet ready for production use." That got us 0.99.0,
> > 0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
> > continuing this pattern would be done with 1.99, I suppose.
> >
> > This issue I take with this approach is that back before 1.0, we could
> > count on users thinking of 0.99 as a different major release train
> > than 0.98. Now, I'm concerned that some might lump 1.99 in with the
> > 1.y major release series.
> >
> > Alternatively we could expressly label the releases as alpha/beta
> > based on our confidence. This would give us 2.0.0-alpha1,
> > 2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
> > futzing with sort order, but clearly conveys that these releases are
> > both part of what will be the 2.y major release series and not for the
> > faint of heart.
> >
> >
> > Thoughts?
> >
>

[jira] [Resolved] (HBASE-17842) Reference book - HBase application integration test is not correct

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser resolved HBASE-17842.

Resolution: Invalid

Agreed, Jan.

[~iraj.hedayati], please feel free to contact Cloudera to report any 
documentation issues on their website. This JIRA project is for Apache HBase.

> Reference book - HBase application integration test is not correct
> --
>
> Key: HBASE-17842
> URL: https://issues.apache.org/jira/browse/HBASE-17842
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Iraj Hedayati
>Priority: Critical
>
> The directions in the following section are not correct.
> https://archive.cloudera.com/cdh5/cdh/5/hbase-1.2.0-cdh5.8.0/book.html#_integration_testing_with_an_hbase_mini_cluster
> It is pointing to HBase 0.98.3 and `HBaseTestingUtility` can not be resolved 
> in 1.2.0



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Josh Elser]

I'm partial to 2.0.0-alpha[x]/beta[x]

* Conveys that it's 2.x (not 1.x)
* Conveys "instability"
* Doesn't buck Maven's view of the world (Maven is happy with a version 
string of 2.0.0-alpha)

* Still enables a "2.0.0" later

Sean Busbey wrote:

Hi folks!

What are folks opinions on how we name releases leading up to HBase
2.0 that aren't quite done yet?

For 1.0, we used 0.99 as a placeholder for "what we expect will be in
1.0 but is not yet ready for production use." That got us 0.99.0,
0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
continuing this pattern would be done with 1.99, I suppose.

This issue I take with this approach is that back before 1.0, we could
count on users thinking of 0.99 as a different major release train
than 0.98. Now, I'm concerned that some might lump 1.99 in with the
1.y major release series.

Alternatively we could expressly label the releases as alpha/beta
based on our confidence. This would give us 2.0.0-alpha1,
2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
futzing with sort order, but clearly conveys that these releases are
both part of what will be the 2.y major release series and not for the
faint of heart.


Thoughts?

Re: [DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Enis Söztutar]

I would automatically -1 any release with a number like 1.99 regardless of
content.

Semantic versioning which we are following already provides an answer for
this:
http://semver.org/#spec-item-9

>From my experience as RM for 0.99.x series and 1.0.x series, I would
suggest we do 2.0.0-alpha1 and alpha2, and one or two betas. I think we
should start the alpha1 release now which does not have to wait for
anything but packaging work.

Enis

On Tue, Mar 28, 2017 at 1:19 PM, Sean Busbey  wrote:

> Hi folks!
>
> What are folks opinions on how we name releases leading up to HBase
> 2.0 that aren't quite done yet?
>
> For 1.0, we used 0.99 as a placeholder for "what we expect will be in
> 1.0 but is not yet ready for production use." That got us 0.99.0,
> 0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
> continuing this pattern would be done with 1.99, I suppose.
>
> This issue I take with this approach is that back before 1.0, we could
> count on users thinking of 0.99 as a different major release train
> than 0.98. Now, I'm concerned that some might lump 1.99 in with the
> 1.y major release series.
>
> Alternatively we could expressly label the releases as alpha/beta
> based on our confidence. This would give us 2.0.0-alpha1,
> 2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
> futzing with sort order, but clearly conveys that these releases are
> both part of what will be the 2.y major release series and not for the
> faint of heart.
>
>
> Thoughts?
>

[DISCUSS] What are we going to call the releases leading up to HBase 2.0?

[Sean Busbey]

Hi folks!

What are folks opinions on how we name releases leading up to HBase
2.0 that aren't quite done yet?

For 1.0, we used 0.99 as a placeholder for "what we expect will be in
1.0 but is not yet ready for production use." That got us 0.99.0,
0.99.1, and 0.99.2 before we declared 1.0.0 ready for use. For 2.0,
continuing this pattern would be done with 1.99, I suppose.

This issue I take with this approach is that back before 1.0, we could
count on users thinking of 0.99 as a different major release train
than 0.98. Now, I'm concerned that some might lump 1.99 in with the
1.y major release series.

Alternatively we could expressly label the releases as alpha/beta
based on our confidence. This would give us 2.0.0-alpha1,
2.0.0-alpha2, etc, 2.0.0-beta1, etc. This has the disadvantage of
futzing with sort order, but clearly conveys that these releases are
both part of what will be the 2.y major release series and not for the
faint of heart.


Thoughts?

HBaseCon2017 Registration is now Open!

[stack]

The HBaseCon West 2017 registration website is up [1]. Registration is
no-charge (Courtesy of our gracious hosts). Space is limited so be sure to
register early (This means YOU!, the HBase Community). Note, HBaseCon will
be in Mountain View @ Google, NOT in San Francisco, on June 12th (The day
before DataWorks Summit, San Jose)

The CfP is still open [2]. It closes April 24th so get those talks in; we'd
love to hear from you.

Yours,
The HBaseCon2017 Program Committee

1. Visit hbasecon.com and you will be redirected to eventbrite
2. https://easychair.org/cfp/hbasecon2017

Re: How threads interact with each other in HBase

[Josh Elser]

1.1 -> 2: don't forget about the block cache which can invalidate the 
need for any HDFS read.


I think you're over-simplifying the write-path quite a bit. I'm not sure 
what you mean by an 'asynchronous write', but that doesn't exist at the 
HBase RPC layer as that would invalidate the consistency guarantees (if 
an RPC returns to the client that data was "put", then it is durable).


Going off of memory (sorry in advance if I misstate something): the 
general way that data is written to the WAL is a "group commit". You 
have many threads all trying to append data to the WAL -- performance 
would be terrible if you serially applied all of these writes. Instead, 
many writes can be accepted and a the caller receives a Future. The 
caller must wait for the Future to complete. What's happening behind the 
scene is that the writes are being bundled together to reduce the number 
of syncs to the WAL ("grouping" the writes together). When one caller's 
future would complete, what really happened is that the write/sync which 
included the caller's update was committed (along with others). All of 
this is happening inside the RS's implementation of accepting an update.


https://github.com/apache/hbase/blob/55d6dcaf877cc5223e679736eb613173229c18be/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L74-L106

杨苏立 Yang Su Li wrote:

The attachment can be found in the following URL:
http://pages.cs.wisc.edu/~suli/hbase.pdf

Sorry for the inconvenience...


On Mon, Mar 27, 2017 at 8:25 PM, Ted Yu  wrote:


Again, attachment didn't come thru.

Is it possible to formulate as google doc ?

Thanks

On Mon, Mar 27, 2017 at 6:19 PM, 杨苏立 Yang Su Li
wrote:


Hi,

I am a graduate student working on scheduling on storage systems, and we
are interested in how different threads in HBase interact with each other
and how it might affect scheduling.

I have written down my understanding on how HBase/HDFS works based on its
current thread architecture (attached). I am wondering if the developers

of

HBase could take a look at it and let me know if anything is incorrect or
inaccurate, or if I have missed anything.

Thanks a lot for your help!

On Wed, Mar 22, 2017 at 3:39 PM, 杨苏立 Yang Su Li
wrote:


Hi,

I am a graduate student working on scheduling on storage systems, and we
are interested in how different threads in HBase interact with each

other

and how it might affect scheduling.

I have written down my understanding on how HBase/HDFS works based on

its

current thread architecture (attached). I am wondering if the

developers of

HBase could take a look at it and let me know if anything is incorrect

or

inaccurate, or if I have missed anything.

Thanks a lot for your help!

--
Suli Yang

Department of Physics
University of Wisconsin Madison

4257 Chamberlin Hall
Madison WI 53703




--
Suli Yang

Department of Physics
University of Wisconsin Madison

4257 Chamberlin Hall
Madison WI 53703

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/548/artifact/website.patch.zip
 | funzip > cb4fac1d18660094908afd6abaace6e441ec6a49.patch
  git fetch
  git checkout -b asf-site-cb4fac1d18660094908afd6abaace6e441ec6a49 
origin/asf-site
  git am --whitespace=fix cb4fac1d18660094908afd6abaace6e441ec6a49.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-cb4fac1d18660094908afd6abaace6e441ec6a49 branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-cb4fac1d18660094908afd6abaace6e441ec6a49:asf-site
  git checkout asf-site
  git branch -D asf-site-cb4fac1d18660094908afd6abaace6e441ec6a49

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/548/console

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/547/artifact/website.patch.zip
 | funzip > ea566e7c541cb18f785b30a3d59a000718b4e079.patch
  git fetch
  git checkout -b asf-site-ea566e7c541cb18f785b30a3d59a000718b4e079 
origin/asf-site
  git am --whitespace=fix ea566e7c541cb18f785b30a3d59a000718b4e079.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-ea566e7c541cb18f785b30a3d59a000718b4e079 branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-ea566e7c541cb18f785b30a3d59a000718b4e079:asf-site
  git checkout asf-site
  git branch -D asf-site-ea566e7c541cb18f785b30a3d59a000718b4e079

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/547/console

Re: [ANNOUNCE] - Welcome our new HBase committer Anastasia Braginsky

[Nick Dimiduk]

Thank you for all your contributions Anastasia!

On Mon, Mar 27, 2017 at 5:38 AM ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi All
>
> Welcome Anastasia Braginsky, one more female committer to HBase. She has
> been active now for a while with her Compacting memstore feature and she
> along with Eshcar have done lot of talks in various meetups and HBaseCon on
> their feature.
>
> Welcome onboard and looking forward to work with you Anastasia !!!
>
> Regards
> Ram
>

[jira] [Created] (HBASE-17843) JUnit test timed out in TestRegionReplicaFailover.java

[Qilin Cao (JIRA)]

Qilin Cao created HBASE-17843:
-

 Summary: JUnit test timed out in TestRegionReplicaFailover.java
 Key: HBASE-17843
 URL: https://issues.apache.org/jira/browse/HBASE-17843
 Project: HBase
  Issue Type: Improvement
Reporter: Qilin Cao
Priority: Trivial


I found junit test failed in TestRegionReplicaFailover.java, so I changed the 
testPrimaryRegionKill method  test timeout = 24
error logs:
Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 285.221 sec <<< 
FAILURE! - in org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
testPrimaryRegionKill[0](org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover)
  Time elapsed: 125.963 sec  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 12 
milliseconds



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)