[jira] [Resolved] (HBASE-18174) Implement Table#checkAndPut()

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-18174.

   Resolution: Fixed
 Assignee: Ted Yu
 Hadoop Flags: Reviewed
Fix Version/s: HBASE-14850

Thanks for the review, Enis.

> Implement Table#checkAndPut()
> -
>
> Key: HBASE-18174
> URL: https://issues.apache.org/jira/browse/HBASE-18174
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: HBASE-14850
>
> Attachments: 18174.v10.txt, 18174.v11.txt, 18174.v12.txt, 
> 18174.v1.txt, 18174.v7.lambda.txt, 18174.v9.txt
>
>
> This task is to implement Table#checkAndPut() method



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HBASE-17594) Add supplemental model for jsr311-api

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved HBASE-17594.
---
   Resolution: Duplicate
 Assignee: (was: Appy)
Fix Version/s: 2.0.0-alpha-2

> Add supplemental model for jsr311-api
> -
>
> Key: HBASE-17594
> URL: https://issues.apache.org/jira/browse/HBASE-17594
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
> Fix For: 2.0.0-alpha-2
>
>
> It's transitive dependency from hadoop 3.
> We don't see any failure for that because for some reason our hadoop3 profile 
> check isn't working, I say so because I wasn't able to do mvn install. (see 
> HBASE-17593)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-18107.
---
Resolution: Won't Fix

Resolving as won't fix. This is old stuff that came in long time ago. It will 
be replaced before we release hbase2. See HBASE-18105.




> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18206) Organize supplemental-models.xml

[Mike Drob (JIRA)]

Mike Drob created HBASE-18206:
-

 Summary: Organize supplemental-models.xml
 Key: HBASE-18206
 URL: https://issues.apache.org/jira/browse/HBASE-18206
 Project: HBase
  Issue Type: Bug
  Components: dependencies
Reporter: Mike Drob


At one point, the supplemental-models.xml file was organized by module and had 
comments on what brings in each dependency.

Since then, I think it has drifted away do to folks adding in dependencies in 
any which place there. It's probably more maintainable to simply alphabetize by 
group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18205) Output multiple license failures in a single run

[Mike Drob (JIRA)]

Mike Drob created HBASE-18205:
-

 Summary: Output multiple license failures in a single run
 Key: HBASE-18205
 URL: https://issues.apache.org/jira/browse/HBASE-18205
 Project: HBase
  Issue Type: Bug
  Components: dependencies
Reporter: Mike Drob






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18204) [C++] Cleanup outgoing RPCs on connection close

[Enis Soztutar (JIRA)]

Enis Soztutar created HBASE-18204:
-

 Summary: [C++] Cleanup outgoing RPCs on connection close
 Key: HBASE-18204
 URL: https://issues.apache.org/jira/browse/HBASE-18204
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar


Our client-dispatcher maintains a map of outgoing RPCs per server with the 
promises. 
In case the server goes down, or TCP connection is closed, we should complete 
the outgoing RPCs with exceptions so that higher level waiters can unblock and 
retry. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18203) Intelligently manage a pool of open references to store files

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-18203:
--

 Summary: Intelligently manage a pool of open references to store 
files
 Key: HBASE-18203
 URL: https://issues.apache.org/jira/browse/HBASE-18203
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 2.0.0
Reporter: Andrew Purtell


When bringing a region online we open every store file and keep the file open, 
to avoid further round trips to the HDFS namenode during reads. Naively keeping 
open every store file we encounter is a bad idea. There should be an upper 
bound. We should close and reopen files as needed once we are above the upper 
bound. We should choose candidates to close on a LRU basis. Otherwise we can 
(and some users have in production) overrun high (~64k) open file handle limits 
on the server if the aggregate number of store files is too large. 

Note the 'open files' here refers to open/active references to files at the 
HDFS level. How this maps to active file descriptors at the OS level depends on 
concurrency of access (block transfers, short circuit reads). The more open 
files we have at the HDFS level the higher number of OS level file handles we 
can expect to consume.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18202) Trim down supplemental models file for unnecessary entries

[Mike Drob (JIRA)]

Mike Drob created HBASE-18202:
-

 Summary: Trim down supplemental models file for unnecessary entries
 Key: HBASE-18202
 URL: https://issues.apache.org/jira/browse/HBASE-18202
 Project: HBase
  Issue Type: Task
  Components: dependencies
Reporter: Mike Drob


With the more permissive "Apache License" check in HBASE-18033, we can remove 
many entries from the supplemental-models.xml file. This issue is to track that 
work separately.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [ANNOUNCE] New HBase committer Allan Yang

[Esteban Gutierrez]

Congrats, Allan!


--
Cloudera, Inc.


On Fri, Jun 9, 2017 at 10:06 AM, Andrew Purtell  wrote:

> Congratulations and welcome, Allan!
>
> On Thu, Jun 8, 2017 at 8:49 PM, Yu Li  wrote:
>
> > On behalf of the Apache HBase PMC, I am pleased to announce that Allan
> Yang
> > has accepted the PMC's invitation to become a committer on the
> > project. We appreciate all of Allan's generous contributions thus far and
> > look forward to his continued involvement.
> >
> > Congratulations and welcome, Allan!
> >
>
>
>
> --
> Best regards,
>
>- Andy
>
> If you are given a choice, you believe you have acted freely. - Raymond
> Teller (via Peter Watts)
>

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Andrew Purtell]

+1

On Thu, Jun 8, 2017 at 12:33 AM, Stack  wrote:

> The first release candidate for HBase 2.0.0-alpha-1 is up at:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-1RC0/
>
> Maven artifacts are available from a staging directory here:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1169
>
> All was signed w/ my key 8ACC93D2
> 
>
> I tagged the RC as 2.0.0-alpha-1RC0
> (c830a0f47f58d4892dd3300032c8244d6278aecc).
>
> hbase-2.0.0-alpha-1 will be our first 2.0.0 release. It is a rough cut
> ('alpha') not-for-production preview of what hbase-2.0.0 will look like. It
> is what we used to call a 'Developer' release[1] meant mostly for devs and
> downstreamers to test drive and flag us early if we there are issues we’ve
> missed ahead of our rolling a production-worthy release.
>
> hbase-2.0.0 includes a fleet of new features that include a new assignment
> manager, means for keeping read and write path off-heap, in-memory
> compactions, and more. I have been keeping a running doc on the state of
> 2.0.0 here [2]. There is much to do still (see aforementioned doc).
>
> The list of features addressed in 2.0.0 so far can be found here [4]. There
> are about 2500. The list of ~500 fixes in 2.0.0 exclusively can be found
> here [3].
>
> Please take it for a spin and vote on whether it ok to put out as our first
> alpha (bar is low for an 'alpha'). Let the VOTE be open for 24 hours.
>
> Thanks,
> St.Ack
>
> 1. http://hbase.apache.org/book.html#hbase.versioning.pre10
> 2.
> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> ktczrlKHK8N4SZzs/edit#heading=h.v21r9nz8g01j
> 3.
> https://issues.apache.org/jira/browse/HBASE-17852?jql=
> project%20%3D%20HBASE%20%20and%20fixVersion%20%3D%202.
> 0.0%20and%20fixVersion%20not%20in%20(1.0.0%2C%201.0.1%2C%
> 201.0.2%2C%201.0.3%2C%201.0.4%2C%201.0.5%2C%201.0.6%2C%201.
> 1.0%2C%201.1.1%2C%201.1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%
> 2C%201.1.6%2C%201.1.7%2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%
> 201.2.0%2C%201.2.1%2C%201.2.2%2C%201.2.3%2C%201.2.4%2C%201.
> 2.5%2C%201.2.6%2C%201.3.0%2C%201.3.1%2C%201.4.0)%20and%20%
> 20(status%20%3D%20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> 4.
> https://issues.apache.org/jira/browse/HBASE-18191?jql=
> project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> 202.0.0)%20and%20(status%20%3D%20Resolved)
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

Re: [ANNOUNCE] New HBase committer Allan Yang

[Andrew Purtell]

Congratulations and welcome, Allan!

On Thu, Jun 8, 2017 at 8:49 PM, Yu Li  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Allan Yang
> has accepted the PMC's invitation to become a committer on the
> project. We appreciate all of Allan's generous contributions thus far and
> look forward to his continued involvement.
>
> Congratulations and welcome, Allan!
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Ted Yu]

+1

On Fri, Jun 9, 2017 at 9:16 AM, Stack  wrote:

> +1 by me.
> St.Ack
>
> On Thu, Jun 8, 2017 at 12:33 AM, Stack  wrote:
>
> > The first release candidate for HBase 2.0.0-alpha-1 is up at:
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-1RC0/
> >
> > Maven artifacts are available from a staging directory here:
> >
> >  https://repository.apache.org/content/repositories/orgapachehbase-1169
> >
> > All was signed w/ my key 8ACC93D2
> > 
> >
> > I tagged the RC as 2.0.0-alpha-1RC0 (c830a0f47f58d4892dd3300032c824
> > 4d6278aecc).
> >
> > hbase-2.0.0-alpha-1 will be our first 2.0.0 release. It is a rough cut
> > ('alpha') not-for-production preview of what hbase-2.0.0 will look like.
> It
> > is what we used to call a 'Developer' release[1] meant mostly for devs
> and
> > downstreamers to test drive and flag us early if we there are issues
> we’ve
> > missed ahead of our rolling a production-worthy release.
> >
> > hbase-2.0.0 includes a fleet of new features that include a new
> assignment
> > manager, means for keeping read and write path off-heap, in-memory
> > compactions, and more. I have been keeping a running doc on the state of
> > 2.0.0 here [2]. There is much to do still (see aforementioned doc).
> >
> > The list of features addressed in 2.0.0 so far can be found here [4].
> > There are about 2500. The list of ~500 fixes in 2.0.0 exclusively can be
> > found here [3].
> >
> > Please take it for a spin and vote on whether it ok to put out as our
> > first alpha (bar is low for an 'alpha'). Let the VOTE be open for 24
> hours.
> >
> > Thanks,
> > St.Ack
> >
> > 1. http://hbase.apache.org/book.html#hbase.versioning.pre10
> > 2. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> > ktczrlKHK8N4SZzs/edit#heading=h.v21r9nz8g01j
> > 3. https://issues.apache.org/jira/browse/HBASE-17852?jql=
> > project%20%3D%20HBASE%20%20and%20fixVersion%20%3D%202.
> > 0.0%20and%20fixVersion%20not%20in%20(1.0.0%2C%201.0.1%2C%
> > 201.0.2%2C%201.0.3%2C%201.0.4%2C%201.0.5%2C%201.0.6%2C%201.
> > 1.0%2C%201.1.1%2C%201.1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%
> > 2C%201.1.6%2C%201.1.7%2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%
> > 201.2.0%2C%201.2.1%2C%201.2.2%2C%201.2.3%2C%201.2.4%2C%201.
> > 2.5%2C%201.2.6%2C%201.3.0%2C%201.3.1%2C%201.4.0)%20and%20%
> > 20(status%20%3D%20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> > 4. https://issues.apache.org/jira/browse/HBASE-18191?jql=
> > project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> > 202.0.0)%20and%20(status%20%3D%20Resolved)
> >
> >
>

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Stack]

+1 by me.
St.Ack

On Thu, Jun 8, 2017 at 12:33 AM, Stack  wrote:

> The first release candidate for HBase 2.0.0-alpha-1 is up at:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-1RC0/
>
> Maven artifacts are available from a staging directory here:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1169
>
> All was signed w/ my key 8ACC93D2
> 
>
> I tagged the RC as 2.0.0-alpha-1RC0 (c830a0f47f58d4892dd3300032c824
> 4d6278aecc).
>
> hbase-2.0.0-alpha-1 will be our first 2.0.0 release. It is a rough cut
> ('alpha') not-for-production preview of what hbase-2.0.0 will look like. It
> is what we used to call a 'Developer' release[1] meant mostly for devs and
> downstreamers to test drive and flag us early if we there are issues we’ve
> missed ahead of our rolling a production-worthy release.
>
> hbase-2.0.0 includes a fleet of new features that include a new assignment
> manager, means for keeping read and write path off-heap, in-memory
> compactions, and more. I have been keeping a running doc on the state of
> 2.0.0 here [2]. There is much to do still (see aforementioned doc).
>
> The list of features addressed in 2.0.0 so far can be found here [4].
> There are about 2500. The list of ~500 fixes in 2.0.0 exclusively can be
> found here [3].
>
> Please take it for a spin and vote on whether it ok to put out as our
> first alpha (bar is low for an 'alpha'). Let the VOTE be open for 24 hours.
>
> Thanks,
> St.Ack
>
> 1. http://hbase.apache.org/book.html#hbase.versioning.pre10
> 2. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> ktczrlKHK8N4SZzs/edit#heading=h.v21r9nz8g01j
> 3. https://issues.apache.org/jira/browse/HBASE-17852?jql=
> project%20%3D%20HBASE%20%20and%20fixVersion%20%3D%202.
> 0.0%20and%20fixVersion%20not%20in%20(1.0.0%2C%201.0.1%2C%
> 201.0.2%2C%201.0.3%2C%201.0.4%2C%201.0.5%2C%201.0.6%2C%201.
> 1.0%2C%201.1.1%2C%201.1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%
> 2C%201.1.6%2C%201.1.7%2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%
> 201.2.0%2C%201.2.1%2C%201.2.2%2C%201.2.3%2C%201.2.4%2C%201.
> 2.5%2C%201.2.6%2C%201.3.0%2C%201.3.1%2C%201.4.0)%20and%20%
> 20(status%20%3D%20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> 4. https://issues.apache.org/jira/browse/HBASE-18191?jql=
> project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> 202.0.0)%20and%20(status%20%3D%20Resolved)
>
>

Re: [ANNOUNCE] New HBase committer Allan Yang

[Dima Spivak]

Yay Allan!

On Thu, Jun 8, 2017 at 8:49 PM Yu Li  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Allan Yang
> has accepted the PMC's invitation to become a committer on the
> project. We appreciate all of Allan's generous contributions thus far and
> look forward to his continued involvement.
>
> Congratulations and welcome, Allan!
>
-- 
-Dima

[jira] [Created] (HBASE-18201) add UT and docs for DataBlockEncodingTool

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-18201:
--

 Summary: add UT and docs for DataBlockEncodingTool
 Key: HBASE-18201
 URL: https://issues.apache.org/jira/browse/HBASE-18201
 Project: HBase
  Issue Type: Task
Reporter: Chia-Ping Tsai
Priority: Minor


There is no example, documents, or tests for DataBlockEncodingTool. We should 
have it friendly if any use case exists. Otherwise, we should just get rid of 
it because DataBlockEncodingTool presumes that the implementation of cell 
returned from DataBlockEncoder is KeyValue. The presume may obstruct the 
cleanup of KeyValue references in the code base of read/write path.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18200) Set hadoop check versions for branch-2 in pre commit

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-18200:
-

 Summary: Set hadoop check versions for branch-2 in pre commit
 Key: HBASE-18200
 URL: https://issues.apache.org/jira/browse/HBASE-18200
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 3.0.0
Reporter: Duo Zhang
 Fix For: 3.0.0


Now it will use the hadoop versions for branch-1.

I do not know how to set the fix versions as the code will be committed to 
master but the branch in trouble is branch-2...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18199) Race in NettyRpcConnection may cause call stuck in BufferCallBeforeInitHandler forever

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-18199:
-

 Summary: Race in NettyRpcConnection may cause call stuck in 
BufferCallBeforeInitHandler forever
 Key: HBASE-18199
 URL: https://issues.apache.org/jira/browse/HBASE-18199
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 3.0.0, 1.4.0, 2.0.0-alpha-1
Reporter: Duo Zhang
Assignee: Duo Zhang
 Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2


Found this when implementing the POC of HDFS-9924.

The problem is that channel.writeAndFlush will find the first outbound handler 
before scheduling the actual write task into event loop, so it is possible that 
when we actually execute the write task, the first outbound handler has already 
been changed but the task still refers to the old handler which has already 
been removed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HBASE-18198) Node creation fails with NPE if ACLs are null

[Bhupendra Kumar Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhupendra Kumar Jain resolved HBASE-18198.
--
Resolution: Invalid

Raised in wrong component. 

> Node creation fails with NPE if ACLs are null
> -
>
> Key: HBASE-18198
> URL: https://issues.apache.org/jira/browse/HBASE-18198
> Project: HBase
>  Issue Type: Bug
>Reporter: Bhupendra Kumar Jain
>
> If null ACLs are passed then zk node creation fails with NPE
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.zookeeper.server.PrepRequestProcessor.removeDuplicates(PrepRequestProcessor.java:1301)
>   at 
> org.apache.zookeeper.server.PrepRequestProcessor.fixupACL(PrepRequestProcessor.java:1341)
>   at 
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(PrepRequestProcessor.java:519)
>   at 
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:1126)
>   at 
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:178)
> {code}
> Below APIs have problem.
> {code}
> public void create(final String path, byte data[], List acl,
> CreateMode createMode, StringCallback cb, Object ctx)
> public void create(final String path, byte data[], List acl,
> CreateMode createMode, Create2Callback cb, Object ctx)
> {code}
> Solution:  Need to handle NULL acl in removeDuplicates method in server.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18198) Node creation fails with NPE if ACLs are null

[Bhupendra Kumar Jain (JIRA)]

Bhupendra Kumar Jain created HBASE-18198:


 Summary: Node creation fails with NPE if ACLs are null
 Key: HBASE-18198
 URL: https://issues.apache.org/jira/browse/HBASE-18198
 Project: HBase
  Issue Type: Bug
Reporter: Bhupendra Kumar Jain


If null ACLs are passed then zk node creation fails with NPE
{code}
java.lang.NullPointerException
at 
org.apache.zookeeper.server.PrepRequestProcessor.removeDuplicates(PrepRequestProcessor.java:1301)
at 
org.apache.zookeeper.server.PrepRequestProcessor.fixupACL(PrepRequestProcessor.java:1341)
at 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(PrepRequestProcessor.java:519)
at 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:1126)
at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:178)
{code}

Below APIs have problem.
{code}
public void create(final String path, byte data[], List acl,
CreateMode createMode, StringCallback cb, Object ctx)

public void create(final String path, byte data[], List acl,
CreateMode createMode, Create2Callback cb, Object ctx)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Proposal for HBASE-16415 (Replication in different namespace)

[Yu Li]

FWIW, taking usage of NamespaceGroupingStrategy introduced in HBASE-14456
together with multiwal may make the design simpler

I think the work is valuable and HBASE-16415 is waiting for its owner, so
just go ahead to take it. More discussion could be done in HBASE-16415 as
Ted suggested.

Thanks.

Best Regards,
Yu

On 9 June 2017 at 06:49, Ted Yu  wrote:

> Mind putting the below proposal on HBASE-16415 ?
>
> Thanks
>
> On Thu, Jun 8, 2017 at 3:24 PM, Jan Kunigk  wrote:
>
> > Hi, with regards to the above JIRA I would like to make the following
> > contribution.
> > I am looking very much forward to feedback and comments.
> >
> > ReplicationSourceWALReaderThread continuously follows WALEntries to be
> > replicated for a specified WAL via WAL.Reader's next() method and adds
> them
> > to WALEntryBatches
> >
> > As far as I can see, those WALEntries are copies of the originally
> > persisted local WALs. In order to direct these Entries to TableNames,
> > different to the source, I propose to intercept the copied WALEntries on
> > the source cluster and probe if they belong to a TableName, which is to
> be
> > re-written.
> >
> > If such a probe is successful, then the WALKey of any such WALEntry needs
> > to be changed accordingly. WALKey provides a getTableName() method, but
> > currently not a setTableName() method, which would simply have to be
> added
> > to change the private TableName member.
> >
> > I propose to intercept the entries via a new method redirectEntry(),
> which
> > is invoked shortly before the entry is added to its WALEntryBatch and
> > immediately after the entry has been filtered by filterEntry() like so:
> >
> > Entry entry = entryStream.next();
> > if (updateSerialReplPos(batch, entry)) {
> >   batch.lastWalPosition = entryStream.getPosition();
> >   break;
> > }
> > entry = filterEntry(entry);
> > entry = redirectEntry(entry); // <--
> > if (entry != null) {
> >   WALEdit edit = entry.getEdit();
> >   if (edit != null && !edit.isEmpty()) {
> > long entrySize = getEntrySize(entry);
> > batch.addEntry(entry);
> >
> > redirectEntry() bases its decisions on a 'Map
> > redirections', where the keys are the source table name and the values
> the
> > destination table name. The Map would be included in the
> > ReplicationPeerConfig, which can be obtained from within
> > ReplicationSourceWALReaderThread via the instance of
> > ReplicationSourceManager, which is in turn passed as an argument to both
> > available constructors.
> >
> > When a TableName object from a WALKey from the WALEntryStream matches the
> > key of any of the entries in the redirections map, that WALKey's
> TableName
> > is replaced by the the value of that entry.
> >
> > The rationale for intercepting on the sending side is that the setup and
> > peer management is performed on the source today already and there is no
> > mechanism I can see which would carry the redirection rules themselves
> > across.
> >
> > Similarly to the way that the hbase shell allows to specify the tables
> and
> > column families to be replicated (set_peer_table_CFs), I propose a new
> > command (also on the sending side) 'set_peer_table_redirections', which
> > accepts a map of Strings, corresponding to the required final
> specification
> > of the redirections as TableNames:
> >
> > set_peer_redirections['ns_source1:table_source1' :
> 'ns_dest1:table_dest1',
> > 'ns_source2:table_source2' : 'ns_dest2:table_dest2', ...
> > 'ns_sourcen:table_sourcen' : 'ns_destn:table_destn', ]
> >
> > Thanks, best, J
> >
>

Re: [ANNOUNCE] New HBase committer Allan Yang

[Allan Yang]

Thank you all!

2017-06-09 14:50 GMT+08:00 ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com>:

> Congratulations !!!
>
> On Fri, Jun 9, 2017 at 12:19 PM, Chia-Ping Tsai 
> wrote:
>
> > Welcome !!!
> >
> > On 2017-06-09 11:49 (+0800), Yu Li  wrote:
> > > On behalf of the Apache HBase PMC, I am pleased to announce that Allan
> > Yang
> > > has accepted the PMC's invitation to become a committer on the
> > > project. We appreciate all of Allan's generous contributions thus far
> and
> > > look forward to his continued involvement.
> > >
> > > Congratulations and welcome, Allan!
> > >
> >
>

[jira] [Resolved] (HBASE-18196) Fix historical FindBugs/Javadoc issues in branch-1.1

[Yu Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li resolved HBASE-18196.
---
Resolution: Duplicate

> Fix historical FindBugs/Javadoc issues in branch-1.1 
> -
>
> Key: HBASE-18196
> URL: https://issues.apache.org/jira/browse/HBASE-18196
> Project: HBase
>  Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Minor
>
> See  
> https://issues.apache.org/jira/browse/HBASE-17678?focusedCommentId=16042572=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042572



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18197) import.java, job output is printing two times.

[Chandra Sekhar (JIRA)]

Chandra Sekhar created HBASE-18197:
--

 Summary: import.java, job output is printing two times.
 Key: HBASE-18197
 URL: https://issues.apache.org/jira/browse/HBASE-18197
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 1.0.2, 1.4.0
Reporter: Chandra Sekhar
Priority: Trivial


import.java, job output is printing two times.
{quote}
after job completed, job.waitForCompletion(true) is calling two times.
{quote}
{code}
Job job = createSubmittableJob(conf, otherArgs);
boolean isJobSuccessful = job.waitForCompletion(true);
if(isJobSuccessful){
  // Flush all the regions of the table
  flushRegionsIfNecessary(conf);
}
long inputRecords = 
job.getCounters().findCounter(TaskCounter.MAP_INPUT_RECORDS).getValue();
long outputRecords = 
job.getCounters().findCounter(TaskCounter.MAP_OUTPUT_RECORDS).getValue();
if (outputRecords < inputRecords) {
  System.err.println("Warning, not all records were imported (maybe 
filtered out).");
  if (outputRecords == 0) {
System.err.println("If the data was exported from HBase 0.94 "+
"consider using -Dhbase.import.version=0.94.");
  }
}

System.exit(job.waitForCompletion(true) ? 0 : 1);
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [ANNOUNCE] New HBase committer Allan Yang

[ramkrishna vasudevan]

Congratulations !!!

On Fri, Jun 9, 2017 at 12:19 PM, Chia-Ping Tsai  wrote:

> Welcome !!!
>
> On 2017-06-09 11:49 (+0800), Yu Li  wrote:
> > On behalf of the Apache HBase PMC, I am pleased to announce that Allan
> Yang
> > has accepted the PMC's invitation to become a committer on the
> > project. We appreciate all of Allan's generous contributions thus far and
> > look forward to his continued involvement.
> >
> > Congratulations and welcome, Allan!
> >
>

Re: [ANNOUNCE] New HBase committer Allan Yang

[Chia-Ping Tsai]

Welcome !!!

On 2017-06-09 11:49 (+0800), Yu Li  wrote: 
> On behalf of the Apache HBase PMC, I am pleased to announce that Allan Yang
> has accepted the PMC's invitation to become a committer on the
> project. We appreciate all of Allan's generous contributions thus far and
> look forward to his continued involvement.
> 
> Congratulations and welcome, Allan!
>