Re: Moving 2.0 forward

[Guanghao Zhang]

About compatibility, there is one incompatible change about the replication
TableCFs' config. The old config is a string and it concatenate the list of
tables and column families in format "table1:cf1,cf2;table2:cfA,cfB" in
zookeeper for table-cf to replication peer mapping. When parse the config,
it use ":" to split the string. If table name includes namespace, it will
be wrong (See HBASE-11386
). It is a problem since
we support namespace (0.98). So HBASE-11393
 (and HBASE-16653
) changed it to a PB
object. When rolling update cluster, you need rolling master first. And the
master will try to translate the string config to a PB object. But there
are two problems.
1. Permission problem. The replication client can write the zookeeper
directly. So the znode may have different owner. And master may don't have
the write permission for the znode. It maybe failed to translate old
table-cfs string to new PB Object. See HBASE-16938

2. We usually keep compatibility between old client and new server. But the
old replication client may write a string config to znode directly. Then
the new server can't parse them.

PS: HBASE-11386 is a problem since 0.98. But there are not too many users
report this problem. So it maybe not a big incompatible issue..

2017-08-15 1:54 GMT+08:00 Stack :

> Heads-up:
>
> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme is
> updated dependencies, reliance on relocated popular libs (guava, netty,
> protobuf), purge of checked-in generated src, and master-carries-no-regions
> by default.
>
> alpha3 I hope will follow soon after (end-of-August?). Its theme will be
> settling the APIs and compatibility (At first blush, we are not looking too
> bad; our Sean ran some tests over weekend that have hbase-1 client running
> against an hbase-2 cluster). The Coprocessor Interface revamp should be
> done by alpha3 (i.e. returning Interfaces rather than Implementations, and
> our shutdown of CPs accessing classes in hbase marked InterfaceAudience).
> We'll also have purged thirdparty classes from our API; e.g. guava 0.12
> Service showing through in our replication API and protobufs in Admin
> Interface. On alpha3, we will have to do a bunch of outreach to make sure
> our downstreamers are up on what is coming down the pipe.
>
> Beta1 in mid-September?
>
> I encourage you to check out the items marked for hbase2:
> https://issues.apache.org/jira/projects/HBASE/versions/12327188 Edit as
> you
> see appropriate. Punt if you know the JIRA will not get any attention in
> next month or so.
>
> A bunch of issues marked blocker are unassigned. I'll leave them as is
> another while but I'll boot them soon.
>
> While I have your attention:
>
> + I think we should leave thrift version at 0.9.3. Moving hbase thrift to
> 0.10.0 will break existing clients. The change is easy enough if folks need
> to upgrade their hbase thrift. See HBASE-18591.
> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first (0.98?).
>
> St.Ack
>
>
>
> On Wed, Aug 2, 2017 at 9:43 AM, Stack  wrote:
>
> >
> >
> > On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser  wrote:
> >
> >>
> >>
> >> On 7/31/17 9:00 AM, Stack wrote:
> >>
> >>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser
> wrote:
> >>>
> >>> ...
> 
>  I like the idea of this also hitting 2.0 as it would make the feature
> a
>  bit more "real", but am obviously a little nervous (I have no reason
> to
>  be
>  nervous though). I am pretty happy with the feature in terms of how
>  much it
>  is covered via testing.
> 
>  https://issues.apache.org/jira/browse/HBASE-17748
> 
> 
> 
>  Sounds good to me. Whats involved? Backport? If so, +1 Josh.
> >>>
> >>> Last think on space quota says that need doc too. See 'Space Quota' in
> >>> here:
> >>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> >>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
> >>> Does this little section need an update Josh?
> >>>
> >>> Thanks,
> >>> S
> >>>
> >>
> >> Yep, just a couple of cherry-picks. Good test coverage and some docs
> >> already included for 17748.  Happy to put that on my plate if you're
> good
> >> with it. I can reasonably assume that no one is against it :)
> >>
> >> I think I had knocked out docs for the "phase 1" stuff before we merged
> >> it in from the original feature branch. I'll double check and update the
> >> gdoc. Perhaps this was just a timing thing.
> >>
> >
> > Thanks Josh,
> > S
> >
>

[jira] [Created] (HBASE-18617) FuzzyRowKeyFilter should not modify the filter pairs

[vinisha (JIRA)]

vinisha created HBASE-18617:
---

 Summary: FuzzyRowKeyFilter should not modify the filter pairs
 Key: HBASE-18617
 URL: https://issues.apache.org/jira/browse/HBASE-18617
 Project: HBase
  Issue Type: Bug
Reporter: vinisha


Current implementation of FuzzyRowKeyFilter modifies the provided filter pairs: 
https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java#L89-L90

Filter should not change the filter pair. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18616) Shell warns about already initialized constants at startup

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-18616:
--

 Summary: Shell warns about already initialized constants at startup
 Key: HBASE-18616
 URL: https://issues.apache.org/jira/browse/HBASE-18616
 Project: HBase
  Issue Type: Bug
  Components: shell
Reporter: Andrew Purtell
Priority: Minor
 Fix For: 1.4.0, 1.5.0


Fix these warnings printed by the shell at startup:

{noformat}
hbase.rb:82 warning: already initialized constant ENDPOINT_CLASSNAME
hbase.rb:83 warning: already initialized constant CLUSTER_KEY
hbase.rb:84 warning: already initialized constant TABLE_CFS
hbase.rb:86 warning: already initialized constant CONFIG
hbase.rb:87 warning: already initialized constant DATA
{noformat}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18615) hbase-rest tests fail in hbase-2.0.0-alpha2

[stack (JIRA)]

stack created HBASE-18615:
-

 Summary: hbase-rest tests fail in hbase-2.0.0-alpha2
 Key: HBASE-18615
 URL: https://issues.apache.org/jira/browse/HBASE-18615
 Project: HBase
  Issue Type: Bug
Reporter: stack


Pointed out by Andrew on VOTE mail



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Stack]

On Wed, Aug 16, 2017 at 4:21 PM, Andrew Purtell  wrote:

> +1
>
> Checked sums and signatures: ok
> RAT check passes: ok
> Built from source: ok (8u131)
> Unit tests pass: mostly, but hbase-rest is fairly broken* (8u131)
> Loaded 1M rows with LTT: ok (8u131)
>
>
> * - TestClientClusterStatus.testLiveAndDeadServersStatus:160 expected:<6>
> but was:<5>, and just about all of the hbase-rest units have some failures
> due to assertion errors
>
>
You the man Andrew.

Looks like small tests fail but the bigger ones fail. Filed HBASE-18615.
I'm on it.

St.Ack



> On Wed, Aug 16, 2017 at 1:43 PM, Stack  wrote:
>
> > The first release candidate for HBase 2.0.0-alpha-2 is up at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
> >
> > Maven artifacts are available from a staging directory here:
> >
> >   https://repository.apache.org/content/repositories/orgapachehbase-1173
> >
> > All was signed with my key at y 8ACC93D2
> > 
> >
> > I tagged the RC as 2.0.0-alpha-2RC0
> > (7149f999786b6fd5a3fc1f7aec1214afb738925e)
> >
> > hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
> > manager, offheap read/write path, in-memory compactions, etc.) plus a
> > return to branch-1 deploy mode where master carries no regions (branch-2
> > and master up to this had master carrying system tables), a purge of all
> > checked-in generated content, updates to all dependencies including a
> move
> > to rely on new hbase-thirdparty lib for relocated versions of guava,
> netty,
> > and protobuf, etc.
> >
> > This is a rough cut ('alpha'), not-for-production preview of what
> > hbase-2.0.0 will look like meant for devs and downstreamers to test drive
> > and flag us early if we messed up anything ahead of our rolling GAs.
> >
> > The list of features addressed in 2.0.0 so far can be found here [4].
> There
> > are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
> > here [3].
> >
> > I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
> > label [2] has undergone extensive weeding and presents a fairly good
> > picture on what is yet to do before we 2.0.0. Check it out.
> >
> > Please take it for a spin and vote on whether it ok to put out as our
> first
> > alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
> > (Saturday)
> >
> > Thanks,
> > St.Ack
> >
> > 1.
> > https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> > Eu_ktczrlKHK8N4SZzs/edit#
> > 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
> > 3.
> > https://issues.apache.org/jira/browse/HBASE-18611?jql=projec
> > t%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.0.0%20%
> > 7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%20fixVersi
> > on%20%3D%202.0.0-alpha-2)%20and%20fixVersion%20not%20in%
> > 20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%201.0.4%2C%
> > 201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.1.2%2C%201.
> > 1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%2C%201.1.8%
> > 2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%201.2.2%2C%
> > 201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.3.0%2C%201.
> > 3.1%2C%201.4.0)%20and%20%20(status%20%3D%20Open%20or%
> > 20status%20%3D%20%22Patch%20Available%22)
> > 4.
> > https://issues.apache.org/jira/browse/HBASE-18599?jql=projec
> > t%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%202.0.0%20%
> > 7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%7C%
> > 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Andrew Purtell]

+1

Checked sums and signatures: ok
RAT check passes: ok
Built from source: ok (8u131)
Unit tests pass: mostly, but hbase-rest is fairly broken* (8u131)
Loaded 1M rows with LTT: ok (8u131)


* - TestClientClusterStatus.testLiveAndDeadServersStatus:160 expected:<6>
but was:<5>, and just about all of the hbase-rest units have some failures
due to assertion errors

On Wed, Aug 16, 2017 at 1:43 PM, Stack  wrote:

> The first release candidate for HBase 2.0.0-alpha-2 is up at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
>
> Maven artifacts are available from a staging directory here:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1173
>
> All was signed with my key at y 8ACC93D2
> 
>
> I tagged the RC as 2.0.0-alpha-2RC0
> (7149f999786b6fd5a3fc1f7aec1214afb738925e)
>
> hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
> manager, offheap read/write path, in-memory compactions, etc.) plus a
> return to branch-1 deploy mode where master carries no regions (branch-2
> and master up to this had master carrying system tables), a purge of all
> checked-in generated content, updates to all dependencies including a move
> to rely on new hbase-thirdparty lib for relocated versions of guava, netty,
> and protobuf, etc.
>
> This is a rough cut ('alpha'), not-for-production preview of what
> hbase-2.0.0 will look like meant for devs and downstreamers to test drive
> and flag us early if we messed up anything ahead of our rolling GAs.
>
> The list of features addressed in 2.0.0 so far can be found here [4]. There
> are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
> here [3].
>
> I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
> label [2] has undergone extensive weeding and presents a fairly good
> picture on what is yet to do before we 2.0.0. Check it out.
>
> Please take it for a spin and vote on whether it ok to put out as our first
> alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
> (Saturday)
>
> Thanks,
> St.Ack
>
> 1.
> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> Eu_ktczrlKHK8N4SZzs/edit#
> 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
> 3.
> https://issues.apache.org/jira/browse/HBASE-18611?jql=projec
> t%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.0.0%20%
> 7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%20fixVersi
> on%20%3D%202.0.0-alpha-2)%20and%20fixVersion%20not%20in%
> 20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%201.0.4%2C%
> 201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.1.2%2C%201.
> 1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%2C%201.1.8%
> 2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%201.2.2%2C%
> 201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.3.0%2C%201.
> 3.1%2C%201.4.0)%20and%20%20(status%20%3D%20Open%20or%
> 20status%20%3D%20%22Patch%20Available%22)
> 4.
> https://issues.apache.org/jira/browse/HBASE-18599?jql=projec
> t%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%202.0.0%20%
> 7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%7C%
> 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[jira] [Resolved] (HBASE-18613) Race condition between master restart and test code when restoring distributed cluster after integration test

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser resolved HBASE-18613.

Resolution: Invalid

Jumped the gun on this one. That warning wasn't actually fatal. The test had 
failed due to some exceptions in the action a bit earlier in the log.

> Race condition between master restart and test code when restoring 
> distributed cluster after integration test
> -
>
> Key: HBASE-18613
> URL: https://issues.apache.org/jira/browse/HBASE-18613
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13
>
>
> Noticed the following in some internal testing (line numbers likely are 
> skewed)
> {noformat}
> 2017-08-16 21:20:25,557| 2017-08-16 21:20:25,553 WARN  [main] 
> client.ConnectionManager$HConnectionImplementation: Checking master connection
> 2017-08-16 21:20:25,557| com.google.protobuf.ServiceException: 
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to 
> master1.domain.com/10.0.2.131:16000 failed on local exception: 
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to 
> master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
> 2017-08-16 21:20:25,557| at 
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
> 2017-08-16 21:20:25,558| at 
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> 2017-08-16 21:20:25,560| at 
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:62739)
> 2017-08-16 21:20:25,560| at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionManager.java:1448)
> 2017-08-16 21:20:25,561| at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionManag
> er.java:2124)
> 2017-08-16 21:20:25,561| at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1712)
> 2017-08-16 21:20:25,562| at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getMaster(ConnectionManager.java:1701)
> 2017-08-16 21:20:25,562| at 
> org.apache.hadoop.hbase.DistributedHBaseCluster.getMasterAdminService(DistributedHBaseCluster.java:153)
> 2017-08-16 21:20:25,563| at 
> org.apache.hadoop.hbase.DistributedHBaseCluster.waitForActiveAndReadyMaster(DistributedHBaseCluster.java:184)
> 2017-08-16 21:20:25,563| at 
> org.apache.hadoop.hbase.HBaseCluster.waitForActiveAndReadyMaster(HBaseCluster.java:204)
> 2017-08-16 21:20:25,563| at 
> org.apache.hadoop.hbase.DistributedHBaseCluster.restoreMasters(DistributedHBaseCluster.java:278)
> 2017-08-16 21:20:25,563| at 
> org.apache.hadoop.hbase.DistributedHBaseCluster.restoreClusterStatus(DistributedHBaseCluster.java:239)
> 2017-08-16 21:20:25,563| at 
> org.apache.hadoop.hbase.HBaseCluster.restoreInitialStatus(HBaseCluster.java:235)
> 2017-08-16 21:20:25,564| at 
> org.apache.hadoop.hbase.IntegrationTestingUtility.restoreCluster(IntegrationTestingUtility.java:99)
> 2017-08-16 21:20:25,564| at 
> org.apache.hadoop.hbase.IntegrationTestBase.cleanUpCluster(IntegrationTestBase.java:200)
> 2017-08-16 21:20:25,564| at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.cleanUpCluster(IntegrationTestDDLMasterFailover.java:146)
> 2017-08-16 21:20:25,564| at 
> org.apache.hadoop.hbase.IntegrationTestBase.cleanUp(IntegrationTestBase.java:140)
> 2017-08-16 21:20:25,564| at 
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:125)
> 2017-08-16 21:20:25,565| at 
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> 2017-08-16 21:20:25,565| at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> 2017-08-16 21:20:25,565| at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.main(IntegrationTestDDLMasterFailover.java:832)
> 2017-08-16 21:20:25,566| Caused by: 
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to 
> master1.domain.com/10.0.2.131:16000 failed on local exception: 
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to 
> master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
> 2017-08-16 21:20:25,566| at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1258)
> 2017-08-16 21:20:25,566| at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1229)
> 2017-08-16 

[jira] [Created] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI

[Biju Nair (JIRA)]

Biju Nair created HBASE-18614:
-

 Summary: Setting BUCKET_CACHE_COMBINED_KEY to false disables stats 
on RS UI
 Key: HBASE-18614
 URL: https://issues.apache.org/jira/browse/HBASE-18614
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 1.1.2
Reporter: Biju Nair


Enabling offheap cache and setting {{BUCKET_CACHE_COMBINED_KEY}} to {{false}} 
in site xml to make offheap cache a strict L2 cache to LRU cache, disables the 
L2 stats normally rendered on region sesrver UI.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Mike Drob]

Never mind, I see the files there now. Thanks for making this, Stack!

On Wed, Aug 16, 2017 at 3:49 PM, Mike Drob  wrote:

> I get a 404 when I try to go to the dist.a.o link. Are we waiting on
> mirrors to update, or any uploads to finish?
>
> On Wed, Aug 16, 2017 at 3:43 PM, Stack  wrote:
>
>> The first release candidate for HBase 2.0.0-alpha-2 is up at:
>>
>>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
>>
>> Maven artifacts are available from a staging directory here:
>>
>>   https://repository.apache.org/content/repositories/orgapachehbase-1173
>>
>> All was signed with my key at y 8ACC93D2
>> 
>>
>> I tagged the RC as 2.0.0-alpha-2RC0
>> (7149f999786b6fd5a3fc1f7aec1214afb738925e)
>>
>> hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
>> manager, offheap read/write path, in-memory compactions, etc.) plus a
>> return to branch-1 deploy mode where master carries no regions (branch-2
>> and master up to this had master carrying system tables), a purge of all
>> checked-in generated content, updates to all dependencies including a move
>> to rely on new hbase-thirdparty lib for relocated versions of guava,
>> netty,
>> and protobuf, etc.
>>
>> This is a rough cut ('alpha'), not-for-production preview of what
>> hbase-2.0.0 will look like meant for devs and downstreamers to test drive
>> and flag us early if we messed up anything ahead of our rolling GAs.
>>
>> The list of features addressed in 2.0.0 so far can be found here [4].
>> There
>> are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
>> here [3].
>>
>> I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
>> label [2] has undergone extensive weeding and presents a fairly good
>> picture on what is yet to do before we 2.0.0. Check it out.
>>
>> Please take it for a spin and vote on whether it ok to put out as our
>> first
>> alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
>> (Saturday)
>>
>> Thanks,
>> St.Ack
>>
>> 1.
>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>> Eu_ktczrlKHK8N4SZzs/edit#
>> 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
>> 3.
>> https://issues.apache.org/jira/browse/HBASE-18611?jql=projec
>> t%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.0.0%20%
>> 7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%20fixVersi
>> on%20%3D%202.0.0-alpha-2)%20and%20fixVersion%20not%20in%
>> 20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%201.0.4%2C%
>> 201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.1.2%2C%201.
>> 1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%2C%201.1.8%
>> 2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%201.2.2%2C%
>> 201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.3.0%2C%201.
>> 3.1%2C%201.4.0)%20and%20%20(status%20%3D%20Open%20or%
>> 20status%20%3D%20%22Patch%20Available%22)
>> 4.
>> https://issues.apache.org/jira/browse/HBASE-18599?jql=projec
>> t%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%202.0.0%20%
>> 7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%7C%
>> 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
>>
>
>

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Stack]

On Wed, Aug 16, 2017 at 1:49 PM, Mike Drob  wrote:

> I get a 404 when I try to go to the dist.a.o link. Are we waiting on
> mirrors to update, or any uploads to finish?
>
>
Retry? It seems to be there now (It was uploading when I sent the note).
M




> On Wed, Aug 16, 2017 at 3:43 PM, Stack  wrote:
>
> > The first release candidate for HBase 2.0.0-alpha-2 is up at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
> >
> > Maven artifacts are available from a staging directory here:
> >
> >   https://repository.apache.org/content/repositories/orgapachehbase-1173
> >
> > All was signed with my key at y 8ACC93D2
> > 
> >
> > I tagged the RC as 2.0.0-alpha-2RC0
> > (7149f999786b6fd5a3fc1f7aec1214afb738925e)
> >
> > hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
> > manager, offheap read/write path, in-memory compactions, etc.) plus a
> > return to branch-1 deploy mode where master carries no regions (branch-2
> > and master up to this had master carrying system tables), a purge of all
> > checked-in generated content, updates to all dependencies including a
> move
> > to rely on new hbase-thirdparty lib for relocated versions of guava,
> netty,
> > and protobuf, etc.
> >
> > This is a rough cut ('alpha'), not-for-production preview of what
> > hbase-2.0.0 will look like meant for devs and downstreamers to test drive
> > and flag us early if we messed up anything ahead of our rolling GAs.
> >
> > The list of features addressed in 2.0.0 so far can be found here [4].
> There
> > are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
> > here [3].
> >
> > I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
> > label [2] has undergone extensive weeding and presents a fairly good
> > picture on what is yet to do before we 2.0.0. Check it out.
> >
> > Please take it for a spin and vote on whether it ok to put out as our
> first
> > alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
> > (Saturday)
> >
> > Thanks,
> > St.Ack
> >
> > 1.
> > https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> > ktczrlKHK8N4SZzs/edit#
> > 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
> > 3.
> > https://issues.apache.org/jira/browse/HBASE-18611?jql=
> > project%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.
> > 0.0%20%7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%
> > 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20fixVersion%
> > 20not%20in%20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%
> > 201.0.4%2C%201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.
> > 1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%
> > 2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%
> > 201.2.2%2C%201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.
> > 3.0%2C%201.3.1%2C%201.4.0)%20and%20%20(status%20%3D%
> > 20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> > 4.
> > https://issues.apache.org/jira/browse/HBASE-18599?jql=
> > project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> > 202.0.0%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%
> > 7C%20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
> >
>

[jira] [Created] (HBASE-18613) Race condition between master restart and test code when restoring distributed cluster after integration test

[Josh Elser (JIRA)]

Josh Elser created HBASE-18613:
--

 Summary: Race condition between master restart and test code when 
restoring distributed cluster after integration test
 Key: HBASE-18613
 URL: https://issues.apache.org/jira/browse/HBASE-18613
 Project: HBase
  Issue Type: Bug
  Components: integration tests
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Minor
 Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13


Noticed the following in some internal testing (line numbers likely are skewed)

{noformat}
2017-08-16 21:20:25,557| 2017-08-16 21:20:25,553 WARN  [main] 
client.ConnectionManager$HConnectionImplementation: Checking master connection
2017-08-16 21:20:25,557| com.google.protobuf.ServiceException: 
org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to 
master1.domain.com/10.0.2.131:16000 failed on local exception: 
org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to 
master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
2017-08-16 21:20:25,557| at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
2017-08-16 21:20:25,558| at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
2017-08-16 21:20:25,560| at 
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:62739)
2017-08-16 21:20:25,560| at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionManager.java:1448)
2017-08-16 21:20:25,561| at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionManag
er.java:2124)
2017-08-16 21:20:25,561| at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1712)
2017-08-16 21:20:25,562| at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getMaster(ConnectionManager.java:1701)
2017-08-16 21:20:25,562| at 
org.apache.hadoop.hbase.DistributedHBaseCluster.getMasterAdminService(DistributedHBaseCluster.java:153)
2017-08-16 21:20:25,563| at 
org.apache.hadoop.hbase.DistributedHBaseCluster.waitForActiveAndReadyMaster(DistributedHBaseCluster.java:184)
2017-08-16 21:20:25,563| at 
org.apache.hadoop.hbase.HBaseCluster.waitForActiveAndReadyMaster(HBaseCluster.java:204)
2017-08-16 21:20:25,563| at 
org.apache.hadoop.hbase.DistributedHBaseCluster.restoreMasters(DistributedHBaseCluster.java:278)
2017-08-16 21:20:25,563| at 
org.apache.hadoop.hbase.DistributedHBaseCluster.restoreClusterStatus(DistributedHBaseCluster.java:239)
2017-08-16 21:20:25,563| at 
org.apache.hadoop.hbase.HBaseCluster.restoreInitialStatus(HBaseCluster.java:235)
2017-08-16 21:20:25,564| at 
org.apache.hadoop.hbase.IntegrationTestingUtility.restoreCluster(IntegrationTestingUtility.java:99)
2017-08-16 21:20:25,564| at 
org.apache.hadoop.hbase.IntegrationTestBase.cleanUpCluster(IntegrationTestBase.java:200)
2017-08-16 21:20:25,564| at 
org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.cleanUpCluster(IntegrationTestDDLMasterFailover.java:146)
2017-08-16 21:20:25,564| at 
org.apache.hadoop.hbase.IntegrationTestBase.cleanUp(IntegrationTestBase.java:140)
2017-08-16 21:20:25,564| at 
org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:125)
2017-08-16 21:20:25,565| at 
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
2017-08-16 21:20:25,565| at 
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
2017-08-16 21:20:25,565| at 
org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.main(IntegrationTestDDLMasterFailover.java:832)
2017-08-16 21:20:25,566| Caused by: 
org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to 
master1.domain.com/10.0.2.131:16000 failed on local exception: 
org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to 
master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
2017-08-16 21:20:25,566| at 
org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1258)
2017-08-16 21:20:25,566| at 
org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1229)
2017-08-16 21:20:25,566| at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
2017-08-16 21:20:25,566| ... 20 more
2017-08-16 21:20:25,566| Caused by: 
org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to 
master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
2017-08-16 21:20:25,567| at 
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.cleanupCalls(RpcClientImpl.java:1047)
2017-08-16 21:20:25,567| at 

[jira] [Created] (HBASE-18612) Update comparators to be more declarative

[Mike Drob (JIRA)]

Mike Drob created HBASE-18612:
-

 Summary: Update comparators to be more declarative
 Key: HBASE-18612
 URL: https://issues.apache.org/jira/browse/HBASE-18612
 Project: HBase
  Issue Type: Improvement
Reporter: Mike Drob
Assignee: Mike Drob


We can write less code if we use factory methods present on Comparator to build 
the chains. Also has the advantage of being easier to update in the future and 
hopefully easier for incoming folks to understand.

See also: 
https://praveer09.github.io/technology/2016/06/21/writing-comparators-the-java8-way/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Mike Drob]

I get a 404 when I try to go to the dist.a.o link. Are we waiting on
mirrors to update, or any uploads to finish?

On Wed, Aug 16, 2017 at 3:43 PM, Stack  wrote:

> The first release candidate for HBase 2.0.0-alpha-2 is up at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
>
> Maven artifacts are available from a staging directory here:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1173
>
> All was signed with my key at y 8ACC93D2
> 
>
> I tagged the RC as 2.0.0-alpha-2RC0
> (7149f999786b6fd5a3fc1f7aec1214afb738925e)
>
> hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
> manager, offheap read/write path, in-memory compactions, etc.) plus a
> return to branch-1 deploy mode where master carries no regions (branch-2
> and master up to this had master carrying system tables), a purge of all
> checked-in generated content, updates to all dependencies including a move
> to rely on new hbase-thirdparty lib for relocated versions of guava, netty,
> and protobuf, etc.
>
> This is a rough cut ('alpha'), not-for-production preview of what
> hbase-2.0.0 will look like meant for devs and downstreamers to test drive
> and flag us early if we messed up anything ahead of our rolling GAs.
>
> The list of features addressed in 2.0.0 so far can be found here [4]. There
> are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
> here [3].
>
> I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
> label [2] has undergone extensive weeding and presents a fairly good
> picture on what is yet to do before we 2.0.0. Check it out.
>
> Please take it for a spin and vote on whether it ok to put out as our first
> alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
> (Saturday)
>
> Thanks,
> St.Ack
>
> 1.
> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> ktczrlKHK8N4SZzs/edit#
> 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
> 3.
> https://issues.apache.org/jira/browse/HBASE-18611?jql=
> project%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.
> 0.0%20%7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%
> 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20fixVersion%
> 20not%20in%20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%
> 201.0.4%2C%201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.
> 1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%
> 2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%
> 201.2.2%2C%201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.
> 3.0%2C%201.3.1%2C%201.4.0)%20and%20%20(status%20%3D%
> 20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> 4.
> https://issues.apache.org/jira/browse/HBASE-18599?jql=
> project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> 202.0.0%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%
> 7C%20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
>

[VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Stack]

The first release candidate for HBase 2.0.0-alpha-2 is up at:

  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/

Maven artifacts are available from a staging directory here:

  https://repository.apache.org/content/repositories/orgapachehbase-1173

All was signed with my key at y 8ACC93D2


I tagged the RC as 2.0.0-alpha-2RC0
(7149f999786b6fd5a3fc1f7aec1214afb738925e)

hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
manager, offheap read/write path, in-memory compactions, etc.) plus a
return to branch-1 deploy mode where master carries no regions (branch-2
and master up to this had master carrying system tables), a purge of all
checked-in generated content, updates to all dependencies including a move
to rely on new hbase-thirdparty lib for relocated versions of guava, netty,
and protobuf, etc.

This is a rough cut ('alpha'), not-for-production preview of what
hbase-2.0.0 will look like meant for devs and downstreamers to test drive
and flag us early if we messed up anything ahead of our rolling GAs.

The list of features addressed in 2.0.0 so far can be found here [4]. There
are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
here [3].

I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
label [2] has undergone extensive weeding and presents a fairly good
picture on what is yet to do before we 2.0.0. Check it out.

Please take it for a spin and vote on whether it ok to put out as our first
alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
(Saturday)

Thanks,
St.Ack

1.
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#
2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
3.
https://issues.apache.org/jira/browse/HBASE-18611?jql=project%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.0.0%20%7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-2)%20and%20fixVersion%20not%20in%20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%201.0.4%2C%201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%201.2.2%2C%201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.3.0%2C%201.3.1%2C%201.4.0)%20and%20%20(status%20%3D%20Open%20or%20status%20%3D%20%22Patch%20Available%22)
4.
https://issues.apache.org/jira/browse/HBASE-18599?jql=project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%202.0.0%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)

[jira] [Resolved] (HBASE-18595) Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to 2.0.0-alpha2

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-18595.
---
Resolution: Fixed

Pushed on branch-2.

> Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to 2.0.0-alpha2
> --
>
> Key: HBASE-18595
> URL: https://issues.apache.org/jira/browse/HBASE-18595
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-alpha-2
>
>
> Set version for alpha2



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack reopened HBASE-16668:
---

Reopening. This method is not in the Admin Interface. Any reason for that 
[~syuanjiang]?

> Admin class should have a synchronous Admin#mergeRegions* method 
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in 
> 1.x this was an asynchronous call) and replaced it with {{Future 
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave 
> synchronously.  
> For normal java Futures, we could just call the future's {{get()}} method. 
> Unforutnately, the future this method returns doesn't follow java Future 
> convention and throws Unimplemented operation when a  plain {{get()}} is 
> called and makes the api harder to use and read.  We could make this future 
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves 
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is 
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with 
> async semantics. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-16668.
---
Resolution: Fixed
  Assignee: Stephen Yuan Jiang

Resolving as implemented by Stephen over in HBASE-14552.

> Admin class should have a synchronous Admin#mergeRegions* method 
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in 
> 1.x this was an asynchronous call) and replaced it with {{Future 
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave 
> synchronously.  
> For normal java Futures, we could just call the future's {{get()}} method. 
> Unforutnately, the future this method returns doesn't follow java Future 
> convention and throws Unimplemented operation when a  plain {{get()}} is 
> called and makes the api harder to use and read.  We could make this future 
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves 
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is 
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with 
> async semantics. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-18431.

  Resolution: Fixed
Hadoop Flags: Reviewed

Pushed to branch-1.4 and branch-1

> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18431-branch-1.4.patch, 
> HBASE-18431-branch-1.patch, HBASE-18431-branch-2-WIP.patch
>
>
> There are compatibility concerns with branch-1.4. 
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 23
>   Medium - 9
>   Low - 21
> {noformat}
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 88
>   Medium - 0
>   Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise 
> it's current.
> I'm not generally concerned with added methods. 
> The following methods have been added to Public/Evolving interface Table. 
> Pointing them out in case it merits review.
> \\
> * Abstract method Table.getReadRpcTimeout ( ) has been added to this 
> interface.   No effect.
> * Abstract method Table.getWriteRpcTimeout ( ) has been added to this 
> interface.  No effect.
> * Abstract method Table.setReadRpcTimeout ( int ) has been added to this 
> interface.   No effect.
> * Abstract method Table.setWriteRpcTimeout ( int ) has been added to this 
> interface.
> The Public/Evolving interface Admin has some signature changes equating to 
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> *  Abstract method Admin.snapshot ( String, TableName, 
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been 
> removed from Admin.
> *  Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This 
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe 
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and 
> RegionObserver have changed, equating to removed methods. The first set of 
> changes is due to move of SnapshotDescription from HBaseProtos to 
> SnapshotProtos:
> \\
> * Abstract method MasterObserver.postCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> 

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Andrew Purtell]

Thanks Mike. If htrace goes to the attic I would be inclined to support
removal. (Do we keep in branch-1 and below, remove in 2 and up, and
document the removal?) And we should bring this to the attention of the
Hadoop and Phoenix dev communities as well. Let me write a note to
dev@phoenix.



On Wed, Aug 16, 2017 at 10:08 AM, Mike Drob  wrote:

> HTrace committer reporting for duty...
>
> I think that HTrace is lacking momentum right now. If there's renewed
> interest in it, then somebody really needs to speak up. I've started an
> attic discussion on the list there, based on my recent experiences with the
> project, though.
>
> If you're interested, follow along or chime in at
> https://lists.apache.org/thread.html/ff87eb11bbfc3ca27a83c0ab377459
> 767a46175447f763b1bf052987@%3Cdev.htrace.apache.org%3E
>
> Mike
>
> On Wed, Aug 16, 2017 at 11:25 AM, Andrew Purtell 
> wrote:
>
> > Yes, tracing is built on HTrace so as long as the Hadoop, HBase, and
> > Phoenix builds include the same 'generation' of htrace, spans are
> available
> > at all of those layers.
> >
> > ​I guess the question (and implication) is we have zero confidence that
> > HTrace will ever be any good. That correct? HTrace is dead? If the answer
> > is yes, then I don't have an objection. If the answer is no, or maybe,
> then
> > I object to removing tracing support without a replacement of equal
> > viability.​
> >
> >
> > On Wed, Aug 16, 2017 at 8:37 AM, Stack  wrote:
> >
> > > On Tue, Aug 15, 2017 at 5:23 PM, Sean Busbey 
> > > wrote:
> > >
> > > > Say we replaced it with brave for 2.0, Phoenix tracing through to
> hdfs
> > > > would be just as broken as if we pulled it out entirely, no?
> > > >
> > > >
> > > That is my understanding (Correct me if I am wrong but Phoenix does not
> > > show traces through hdfs [1]).
> > >
> > > St.Ack
> > >
> > >
> > >
> > > 1. https://phoenix.apache.org/tracing.html
> > >
> > >
> > >
> > > > On Aug 15, 2017 7:18 PM, "Nick Dimiduk"  wrote:
> > > >
> > > > > Agreed that ripping with no replacement would be too extreme and
> > short
> > > > > notice for 2.0.
> > > > >
> > > > > On Tue, Aug 15, 2017 at 5:14 PM Andrew Purtell <
> apurt...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > > If Phoenix devs are advertising this feature to their users,
> > > > > > > their contributions re: HBase and HDFS maintenance would be
> well
> > > > > > received.
> > > > > >
> > > > > > The counter point to this more mindfulness on the part of HBase
> > devs
> > > to
> > > > > > interface compatibility impacts would go a long way. See
> > HBASE-18431
> > > > > > and PHOENIX-4076.
> > > > > >
> > > > > > Ripping out entirely the only tracing interfaces available is a
> > more
> > > > > > extreme example, although not without cause. (The state of
> HTrace.)
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk <
> ndimi...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > IMHO, we should be actively pruning half-implemented, testless,
> > > > > ownerless
> > > > > > > code. HTrace queues up well for that chopping block. Long back
> I
> > > > wanted
> > > > > > to
> > > > > > > see this particular feature though, but I cannot volunteer for
> > the
> > > > role
> > > > > > > currently. If Phoenix devs are advertising this feature to
> their
> > > > users,
> > > > > > > their contributions re: HBase and HDFS maintenance would be
> well
> > > > > > received.
> > > > > > >
> > > > > > > I question the choice of HTrace as the tool for the job, given
> > the
> > > > > > relative
> > > > > > > lack of activity in that project vs. the myriad alternatives
> out
> > > > there.
> > > > > > > HTrace has been incubating for 2.5 years, has seen only 5
> > releases,
> > > > > with
> > > > > > no
> > > > > > > release in the last 18 months. I haven't evaluated the field
> in a
> > > > > while,
> > > > > > > but there's probably something with much more mature tooling
> and
> > > > > support.
> > > > > > > For instance, openzipkin/brave has much more activity more
> > > recently.
> > > > > > >
> > > > > > > -n
> > > > > > >
> > > > > > > On Tue, Aug 15, 2017 at 2:28 PM Stack 
> wrote:
> > > > > > >
> > > > > > > > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell <
> > > > > apurt...@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Doc update to indicate HTrace is unfinished is fine.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > We could do this. Update jar and doc it as broke.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > Phoenix uses HTrace to support query tracing. It "works".
> > > > > > > >
> > > > > > > >
> > > > > > > > Nit: I didn't think this was htrace. It was an old version
> > pulled
> > > > > into
> > > > > > > > phoenix world where it lives storing to phoenix table
> reporting
> > > on
> > > > > > > > phoenix-only spans.
> > > > > > > 

[jira] [Created] (HBASE-18611) Copy all tests from o.a.h.h.p.TestProtobufUtil to o.a.h.h.s.p.TestProtobufUtil

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-18611:
--

 Summary: Copy all tests from o.a.h.h.p.TestProtobufUtil to 
o.a.h.h.s.p.TestProtobufUtil
 Key: HBASE-18611
 URL: https://issues.apache.org/jira/browse/HBASE-18611
 Project: HBase
  Issue Type: Task
  Components: Protobufs
Affects Versions: 2.0.0-alpha-1, 3.0.0
Reporter: Chia-Ping Tsai
Priority: Minor
 Fix For: 2.0.0


We copied all codes from o.a.h.h.p.TestProtobufUtil to 
o.a.h.h.s.p.ProtobufUtil, but we ignored the test codes. Let us increase the 
code coverage.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Mike Drob]

HTrace committer reporting for duty...

I think that HTrace is lacking momentum right now. If there's renewed
interest in it, then somebody really needs to speak up. I've started an
attic discussion on the list there, based on my recent experiences with the
project, though.

If you're interested, follow along or chime in at
https://lists.apache.org/thread.html/ff87eb11bbfc3ca27a83c0ab377459767a46175447f763b1bf052987@%3Cdev.htrace.apache.org%3E

Mike

On Wed, Aug 16, 2017 at 11:25 AM, Andrew Purtell 
wrote:

> Yes, tracing is built on HTrace so as long as the Hadoop, HBase, and
> Phoenix builds include the same 'generation' of htrace, spans are available
> at all of those layers.
>
> ​I guess the question (and implication) is we have zero confidence that
> HTrace will ever be any good. That correct? HTrace is dead? If the answer
> is yes, then I don't have an objection. If the answer is no, or maybe, then
> I object to removing tracing support without a replacement of equal
> viability.​
>
>
> On Wed, Aug 16, 2017 at 8:37 AM, Stack  wrote:
>
> > On Tue, Aug 15, 2017 at 5:23 PM, Sean Busbey 
> > wrote:
> >
> > > Say we replaced it with brave for 2.0, Phoenix tracing through to hdfs
> > > would be just as broken as if we pulled it out entirely, no?
> > >
> > >
> > That is my understanding (Correct me if I am wrong but Phoenix does not
> > show traces through hdfs [1]).
> >
> > St.Ack
> >
> >
> >
> > 1. https://phoenix.apache.org/tracing.html
> >
> >
> >
> > > On Aug 15, 2017 7:18 PM, "Nick Dimiduk"  wrote:
> > >
> > > > Agreed that ripping with no replacement would be too extreme and
> short
> > > > notice for 2.0.
> > > >
> > > > On Tue, Aug 15, 2017 at 5:14 PM Andrew Purtell 
> > > > wrote:
> > > >
> > > > > > If Phoenix devs are advertising this feature to their users,
> > > > > > their contributions re: HBase and HDFS maintenance would be well
> > > > > received.
> > > > >
> > > > > The counter point to this more mindfulness on the part of HBase
> devs
> > to
> > > > > interface compatibility impacts would go a long way. See
> HBASE-18431
> > > > > and PHOENIX-4076.
> > > > >
> > > > > Ripping out entirely the only tracing interfaces available is a
> more
> > > > > extreme example, although not without cause. (The state of HTrace.)
> > > > >
> > > > >
> > > > > On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk 
> > > > wrote:
> > > > >
> > > > > > IMHO, we should be actively pruning half-implemented, testless,
> > > > ownerless
> > > > > > code. HTrace queues up well for that chopping block. Long back I
> > > wanted
> > > > > to
> > > > > > see this particular feature though, but I cannot volunteer for
> the
> > > role
> > > > > > currently. If Phoenix devs are advertising this feature to their
> > > users,
> > > > > > their contributions re: HBase and HDFS maintenance would be well
> > > > > received.
> > > > > >
> > > > > > I question the choice of HTrace as the tool for the job, given
> the
> > > > > relative
> > > > > > lack of activity in that project vs. the myriad alternatives out
> > > there.
> > > > > > HTrace has been incubating for 2.5 years, has seen only 5
> releases,
> > > > with
> > > > > no
> > > > > > release in the last 18 months. I haven't evaluated the field in a
> > > > while,
> > > > > > but there's probably something with much more mature tooling and
> > > > support.
> > > > > > For instance, openzipkin/brave has much more activity more
> > recently.
> > > > > >
> > > > > > -n
> > > > > >
> > > > > > On Tue, Aug 15, 2017 at 2:28 PM Stack  wrote:
> > > > > >
> > > > > > > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell <
> > > > apurt...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Doc update to indicate HTrace is unfinished is fine.
> > > > > > > >
> > > > > > > >
> > > > > > > We could do this. Update jar and doc it as broke.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Phoenix uses HTrace to support query tracing. It "works".
> > > > > > >
> > > > > > >
> > > > > > > Nit: I didn't think this was htrace. It was an old version
> pulled
> > > > into
> > > > > > > phoenix world where it lives storing to phoenix table reporting
> > on
> > > > > > > phoenix-only spans.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > A colleague made
> > > > > > > > a contribution to HDFS to fix a case where spans could get
> > > muddied
> > > > > > around
> > > > > > > > the WAL. We are carrying that patch on top of 2.7 for the
> > moment
> > > > but
> > > > > > IIRC
> > > > > > > > it's in 2.8.
> > > > > > > >
> > > > > > > > Removing all of our HBase level spans doesn't make sense to
> me
> > > when
> > > > > > spans
> > > > > > > > at HDFS level will not be removed, nor at the Phoenix level.
> > > > > > >
> > > > > > >
> > > > > > > The spans in hbase are just hanging until they get the htrace-4
> > > > revamp
> > > > > > 

[jira] [Resolved] (HBASE-15630) Improve checksum files for releases for easier verification

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved HBASE-15630.
---
Resolution: Won't Fix

Proposed discussion never happened on the dev list, cleaning out old issues.

> Improve checksum files for releases for easier verification
> ---
>
> Key: HBASE-15630
> URL: https://issues.apache.org/jira/browse/HBASE-15630
> Project: HBase
>  Issue Type: Wish
>Affects Versions: 1.2.1
>Reporter: Christopher Tubbs
>Priority: Trivial
>
> Trying to verify latest release (1.2.1), and I found it a bit inconvenient to 
> parse the *.mds checksum file. The line wrapping, white space, and the 
> general format of the file does not lend itself for easy verification.
> I suggest using the standard "coreutils" format for md5sum, sha*sum, etc., 
> instead: 
> {code}
> # md5
> 3d66c0dd4f38fa881046fe64dd680a7a *hbase-1.2.1-src.tar.gz
> # sha1
> 3666a4829d9a8d9285173bfa8e8d0ff5423a22d6 *hbase-1.2.1-src.tar.gz
> # rmd160
> #fb318e84b6256492cfb990aec2238a64c2da21ad *hbase-1.2.1-src.tar.gz
> # sha224
> 89d341a55069e4875f9e6859737062fd7a4c11596811731c4ba95ca0 
> *hbase-1.2.1-src.tar.gz
> # sha256
> e8000a65e98d4c5db7bab54da99a57209fe4ea777ab41e91ae8ccf7bfa2d50dd 
> *hbase-1.2.1-src.tar.gz
> # sha384
> 49aa0620bf0fbe20bbde66cecabb76b22defb9ee609936edc3952889e6484e55c88f1c93d6258a2eaab4a9d5188b6170
>  *hbase-1.2.1-src.tar.gz
> # sha512
> 28956a35a01ae87e9f733664c52c6fd25f9a60a1ff7047bbf306cd433c2a5b863c9bf05aba1d58792b86eec9943ae00e772c4b76fb81c5d210cf256cd074189b
>  *hbase-1.2.1-src.tar.gz
> {code}
> (comment lines added for humans, but ignored by tools; commented out rmd160, 
> because not a coreutils supported algorithm; binary flag optional, could use 
> another space instead... probably only matters for some dos tools)
> This makes it very easy to verify multiple files and hashes using: {{shasum 
> -c file.mds}} or {{sha1sum -c file.mds}} or {{md5sum -c file.mds}}.
> In addition to the file format change, I suggest these two additional changes:
> 1. Drop rmd160. It's not nearly as popular as the others, and it doesn't lend 
> itself to easy verification (no coreutils equivalent command like md5sum, 
> sha1sum, etc.)
> 2. Concatenate hashes from all files into a single file. This makes it easier 
> to verify all downloads at once.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Andrew Purtell]

Yes, tracing is built on HTrace so as long as the Hadoop, HBase, and
Phoenix builds include the same 'generation' of htrace, spans are available
at all of those layers.

​I guess the question (and implication) is we have zero confidence that
HTrace will ever be any good. That correct? HTrace is dead? If the answer
is yes, then I don't have an objection. If the answer is no, or maybe, then
I object to removing tracing support without a replacement of equal
viability.​


On Wed, Aug 16, 2017 at 8:37 AM, Stack  wrote:

> On Tue, Aug 15, 2017 at 5:23 PM, Sean Busbey 
> wrote:
>
> > Say we replaced it with brave for 2.0, Phoenix tracing through to hdfs
> > would be just as broken as if we pulled it out entirely, no?
> >
> >
> That is my understanding (Correct me if I am wrong but Phoenix does not
> show traces through hdfs [1]).
>
> St.Ack
>
>
>
> 1. https://phoenix.apache.org/tracing.html
>
>
>
> > On Aug 15, 2017 7:18 PM, "Nick Dimiduk"  wrote:
> >
> > > Agreed that ripping with no replacement would be too extreme and short
> > > notice for 2.0.
> > >
> > > On Tue, Aug 15, 2017 at 5:14 PM Andrew Purtell 
> > > wrote:
> > >
> > > > > If Phoenix devs are advertising this feature to their users,
> > > > > their contributions re: HBase and HDFS maintenance would be well
> > > > received.
> > > >
> > > > The counter point to this more mindfulness on the part of HBase devs
> to
> > > > interface compatibility impacts would go a long way. See HBASE-18431
> > > > and PHOENIX-4076.
> > > >
> > > > Ripping out entirely the only tracing interfaces available is a more
> > > > extreme example, although not without cause. (The state of HTrace.)
> > > >
> > > >
> > > > On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk 
> > > wrote:
> > > >
> > > > > IMHO, we should be actively pruning half-implemented, testless,
> > > ownerless
> > > > > code. HTrace queues up well for that chopping block. Long back I
> > wanted
> > > > to
> > > > > see this particular feature though, but I cannot volunteer for the
> > role
> > > > > currently. If Phoenix devs are advertising this feature to their
> > users,
> > > > > their contributions re: HBase and HDFS maintenance would be well
> > > > received.
> > > > >
> > > > > I question the choice of HTrace as the tool for the job, given the
> > > > relative
> > > > > lack of activity in that project vs. the myriad alternatives out
> > there.
> > > > > HTrace has been incubating for 2.5 years, has seen only 5 releases,
> > > with
> > > > no
> > > > > release in the last 18 months. I haven't evaluated the field in a
> > > while,
> > > > > but there's probably something with much more mature tooling and
> > > support.
> > > > > For instance, openzipkin/brave has much more activity more
> recently.
> > > > >
> > > > > -n
> > > > >
> > > > > On Tue, Aug 15, 2017 at 2:28 PM Stack  wrote:
> > > > >
> > > > > > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell <
> > > apurt...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Doc update to indicate HTrace is unfinished is fine.
> > > > > > >
> > > > > > >
> > > > > > We could do this. Update jar and doc it as broke.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Phoenix uses HTrace to support query tracing. It "works".
> > > > > >
> > > > > >
> > > > > > Nit: I didn't think this was htrace. It was an old version pulled
> > > into
> > > > > > phoenix world where it lives storing to phoenix table reporting
> on
> > > > > > phoenix-only spans.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > A colleague made
> > > > > > > a contribution to HDFS to fix a case where spans could get
> > muddied
> > > > > around
> > > > > > > the WAL. We are carrying that patch on top of 2.7 for the
> moment
> > > but
> > > > > IIRC
> > > > > > > it's in 2.8.
> > > > > > >
> > > > > > > Removing all of our HBase level spans doesn't make sense to me
> > when
> > > > > spans
> > > > > > > at HDFS level will not be removed, nor at the Phoenix level.
> > > > > >
> > > > > >
> > > > > > The spans in hbase are just hanging until they get the htrace-4
> > > revamp
> > > > > and
> > > > > > the dots get connected.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > We hollow out
> > > > > > > the middle. I think this is worse than leaving things at a half
> > > > > finished
> > > > > > > state. At least in the half finished state it might get
> finished.
> > > > > > >
> > > > > > >
> > > > > > Nod.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > What are the most pressing issues? I'm up for supporting an
> > > "HTrace"
> > > > > get
> > > > > > > well. I'll need your (community) help for getting patches into
> > > Hadoop
> > > > > if
> > > > > > > need be.
> > > > > > >
> > > > > > >
> > > > > > HTrace needs care and a viewer (zipkin I think now it has come
> back
> > > > > alive).
> > > > > >
> > > > > > HBase needs a couple of narratives wired through.
> > > 

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Nick Dimiduk]

On Tue, Aug 15, 2017 at 5:23 PM Sean Busbey  wrote:

> Say we replaced it with brave for 2.0, Phoenix tracing through to hdfs
> would be just as broken as if we pulled it out entirely, no?


Yep! From the perspective of Phoenix, HTrace API is our public API. We
should consider this as we do other places where 3rd party deps leak into
our API. Just as we removed Guava/Service, I suggest we should remove
HTrace/Span. If we want to support tracing, we should provide our own API
for it, and provide connective tissue for interop with whatever 3rd party
libs our users want to use. Or at least interfaces that downstreamers can
use to implement that connective tissue. HBase Spans -> HTrace Spans would
be a good demonstration of such an API, and give us what we need to restore
Span continuation through HDFS.

All this requires an owner, for which tracing in HBase has none. Maybe our
Tamas has some vested interest in being that owner, having brought up the
topic to start with?

On Aug 15, 2017 7:18 PM, "Nick Dimiduk"  wrote:
>
> > Agreed that ripping with no replacement would be too extreme and short
> > notice for 2.0.
> >
> > On Tue, Aug 15, 2017 at 5:14 PM Andrew Purtell 
> > wrote:
> >
> > > > If Phoenix devs are advertising this feature to their users,
> > > > their contributions re: HBase and HDFS maintenance would be well
> > > received.
> > >
> > > The counter point to this more mindfulness on the part of HBase devs to
> > > interface compatibility impacts would go a long way. See HBASE-18431
> > > and PHOENIX-4076.
> > >
> > > Ripping out entirely the only tracing interfaces available is a more
> > > extreme example, although not without cause. (The state of HTrace.)
> > >
> > >
> > > On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk 
> > wrote:
> > >
> > > > IMHO, we should be actively pruning half-implemented, testless,
> > ownerless
> > > > code. HTrace queues up well for that chopping block. Long back I
> wanted
> > > to
> > > > see this particular feature though, but I cannot volunteer for the
> role
> > > > currently. If Phoenix devs are advertising this feature to their
> users,
> > > > their contributions re: HBase and HDFS maintenance would be well
> > > received.
> > > >
> > > > I question the choice of HTrace as the tool for the job, given the
> > > relative
> > > > lack of activity in that project vs. the myriad alternatives out
> there.
> > > > HTrace has been incubating for 2.5 years, has seen only 5 releases,
> > with
> > > no
> > > > release in the last 18 months. I haven't evaluated the field in a
> > while,
> > > > but there's probably something with much more mature tooling and
> > support.
> > > > For instance, openzipkin/brave has much more activity more recently.
> > > >
> > > > -n
> > > >
> > > > On Tue, Aug 15, 2017 at 2:28 PM Stack  wrote:
> > > >
> > > > > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell <
> > apurt...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Doc update to indicate HTrace is unfinished is fine.
> > > > > >
> > > > > >
> > > > > We could do this. Update jar and doc it as broke.
> > > > >
> > > > >
> > > > >
> > > > > > Phoenix uses HTrace to support query tracing. It "works".
> > > > >
> > > > >
> > > > > Nit: I didn't think this was htrace. It was an old version pulled
> > into
> > > > > phoenix world where it lives storing to phoenix table reporting on
> > > > > phoenix-only spans.
> > > > >
> > > > >
> > > > >
> > > > > > A colleague made
> > > > > > a contribution to HDFS to fix a case where spans could get
> muddied
> > > > around
> > > > > > the WAL. We are carrying that patch on top of 2.7 for the moment
> > but
> > > > IIRC
> > > > > > it's in 2.8.
> > > > > >
> > > > > > Removing all of our HBase level spans doesn't make sense to me
> when
> > > > spans
> > > > > > at HDFS level will not be removed, nor at the Phoenix level.
> > > > >
> > > > >
> > > > > The spans in hbase are just hanging until they get the htrace-4
> > revamp
> > > > and
> > > > > the dots get connected.
> > > > >
> > > > >
> > > > >
> > > > > > We hollow out
> > > > > > the middle. I think this is worse than leaving things at a half
> > > > finished
> > > > > > state. At least in the half finished state it might get finished.
> > > > > >
> > > > > >
> > > > > Nod.
> > > > >
> > > > >
> > > > >
> > > > > > What are the most pressing issues? I'm up for supporting an
> > "HTrace"
> > > > get
> > > > > > well. I'll need your (community) help for getting patches into
> > Hadoop
> > > > if
> > > > > > need be.
> > > > > >
> > > > > >
> > > > > HTrace needs care and a viewer (zipkin I think now it has come back
> > > > alive).
> > > > >
> > > > > HBase needs a couple of narratives wired through.
> > > > >
> > > > > Ditto HDFS.
> > > > >
> > > > > St.Ack
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > On Tue, Aug 15, 2017 at 12:10 PM, Stack 
> wrote:
> > > > > >
> 

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Stack]

On Tue, Aug 15, 2017 at 5:23 PM, Sean Busbey  wrote:

> Say we replaced it with brave for 2.0, Phoenix tracing through to hdfs
> would be just as broken as if we pulled it out entirely, no?
>
>
That is my understanding (Correct me if I am wrong but Phoenix does not
show traces through hdfs [1]).

St.Ack



1. https://phoenix.apache.org/tracing.html



> On Aug 15, 2017 7:18 PM, "Nick Dimiduk"  wrote:
>
> > Agreed that ripping with no replacement would be too extreme and short
> > notice for 2.0.
> >
> > On Tue, Aug 15, 2017 at 5:14 PM Andrew Purtell 
> > wrote:
> >
> > > > If Phoenix devs are advertising this feature to their users,
> > > > their contributions re: HBase and HDFS maintenance would be well
> > > received.
> > >
> > > The counter point to this more mindfulness on the part of HBase devs to
> > > interface compatibility impacts would go a long way. See HBASE-18431
> > > and PHOENIX-4076.
> > >
> > > Ripping out entirely the only tracing interfaces available is a more
> > > extreme example, although not without cause. (The state of HTrace.)
> > >
> > >
> > > On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk 
> > wrote:
> > >
> > > > IMHO, we should be actively pruning half-implemented, testless,
> > ownerless
> > > > code. HTrace queues up well for that chopping block. Long back I
> wanted
> > > to
> > > > see this particular feature though, but I cannot volunteer for the
> role
> > > > currently. If Phoenix devs are advertising this feature to their
> users,
> > > > their contributions re: HBase and HDFS maintenance would be well
> > > received.
> > > >
> > > > I question the choice of HTrace as the tool for the job, given the
> > > relative
> > > > lack of activity in that project vs. the myriad alternatives out
> there.
> > > > HTrace has been incubating for 2.5 years, has seen only 5 releases,
> > with
> > > no
> > > > release in the last 18 months. I haven't evaluated the field in a
> > while,
> > > > but there's probably something with much more mature tooling and
> > support.
> > > > For instance, openzipkin/brave has much more activity more recently.
> > > >
> > > > -n
> > > >
> > > > On Tue, Aug 15, 2017 at 2:28 PM Stack  wrote:
> > > >
> > > > > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell <
> > apurt...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Doc update to indicate HTrace is unfinished is fine.
> > > > > >
> > > > > >
> > > > > We could do this. Update jar and doc it as broke.
> > > > >
> > > > >
> > > > >
> > > > > > Phoenix uses HTrace to support query tracing. It "works".
> > > > >
> > > > >
> > > > > Nit: I didn't think this was htrace. It was an old version pulled
> > into
> > > > > phoenix world where it lives storing to phoenix table reporting on
> > > > > phoenix-only spans.
> > > > >
> > > > >
> > > > >
> > > > > > A colleague made
> > > > > > a contribution to HDFS to fix a case where spans could get
> muddied
> > > > around
> > > > > > the WAL. We are carrying that patch on top of 2.7 for the moment
> > but
> > > > IIRC
> > > > > > it's in 2.8.
> > > > > >
> > > > > > Removing all of our HBase level spans doesn't make sense to me
> when
> > > > spans
> > > > > > at HDFS level will not be removed, nor at the Phoenix level.
> > > > >
> > > > >
> > > > > The spans in hbase are just hanging until they get the htrace-4
> > revamp
> > > > and
> > > > > the dots get connected.
> > > > >
> > > > >
> > > > >
> > > > > > We hollow out
> > > > > > the middle. I think this is worse than leaving things at a half
> > > > finished
> > > > > > state. At least in the half finished state it might get finished.
> > > > > >
> > > > > >
> > > > > Nod.
> > > > >
> > > > >
> > > > >
> > > > > > What are the most pressing issues? I'm up for supporting an
> > "HTrace"
> > > > get
> > > > > > well. I'll need your (community) help for getting patches into
> > Hadoop
> > > > if
> > > > > > need be.
> > > > > >
> > > > > >
> > > > > HTrace needs care and a viewer (zipkin I think now it has come back
> > > > alive).
> > > > >
> > > > > HBase needs a couple of narratives wired through.
> > > > >
> > > > > Ditto HDFS.
> > > > >
> > > > > St.Ack
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > On Tue, Aug 15, 2017 at 12:10 PM, Stack 
> wrote:
> > > > > >
> > > > > > > On Tue, Aug 15, 2017 at 11:22 AM, Andrew Purtell <
> > > > apurt...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > -1 to removing HTrace from HBase.
> > > > > > > >
> > > > > > > > Upgrade to the desired version, that would be fine.
> > > > > > > >
> > > > > > > >
> > > > > > > ​t​
> > > > > > >
> > > > > > > What you thinking Andrew?
> > > > > > >
> > > > > > > Trace is broke whether we upgrade or not. As is, we mislead. We
> > > give
> > > > > off
> > > > > > > the impression that hooking up hbase and hdfs via trace tells a
> > > story
> > > > > > about
> > > > > > > request traversals when 

Re: [DISCUSS] Remove HTrace 3.2.0-incubating from HBase

[Stack]

On Tue, Aug 15, 2017 at 5:10 PM, Nick Dimiduk  wrote:

> IMHO, we should be actively pruning half-implemented, testless, ownerless
> code.


+1

What else should we be looking at purging that fits this category?

St.Ack



> I question the choice of HTrace as the tool for the job, given the relative
> lack of activity in that project vs. the myriad alternatives out there.
> HTrace has been incubating for 2.5 years, has seen only 5 releases, with no
> release in the last 18 months. I haven't evaluated the field in a while,
> but there's probably something with much more mature tooling and support.
> For instance, openzipkin/brave has much more activity more recently.
>
> -n
>
> On Tue, Aug 15, 2017 at 2:28 PM Stack  wrote:
>
> > On Tue, Aug 15, 2017 at 12:38 PM, Andrew Purtell 
> > wrote:
> >
> > > Doc update to indicate HTrace is unfinished is fine.
> > >
> > >
> > We could do this. Update jar and doc it as broke.
> >
> >
> >
> > > Phoenix uses HTrace to support query tracing. It "works".
> >
> >
> > Nit: I didn't think this was htrace. It was an old version pulled into
> > phoenix world where it lives storing to phoenix table reporting on
> > phoenix-only spans.
> >
> >
> >
> > > A colleague made
> > > a contribution to HDFS to fix a case where spans could get muddied
> around
> > > the WAL. We are carrying that patch on top of 2.7 for the moment but
> IIRC
> > > it's in 2.8.
> > >
> > > Removing all of our HBase level spans doesn't make sense to me when
> spans
> > > at HDFS level will not be removed, nor at the Phoenix level.
> >
> >
> > The spans in hbase are just hanging until they get the htrace-4 revamp
> and
> > the dots get connected.
> >
> >
> >
> > > We hollow out
> > > the middle. I think this is worse than leaving things at a half
> finished
> > > state. At least in the half finished state it might get finished.
> > >
> > >
> > Nod.
> >
> >
> >
> > > What are the most pressing issues? I'm up for supporting an "HTrace"
> get
> > > well. I'll need your (community) help for getting patches into Hadoop
> if
> > > need be.
> > >
> > >
> > HTrace needs care and a viewer (zipkin I think now it has come back
> alive).
> >
> > HBase needs a couple of narratives wired through.
> >
> > Ditto HDFS.
> >
> > St.Ack
> >
> >
> >
> > >
> > > On Tue, Aug 15, 2017 at 12:10 PM, Stack  wrote:
> > >
> > > > On Tue, Aug 15, 2017 at 11:22 AM, Andrew Purtell <
> apurt...@apache.org>
> > > > wrote:
> > > >
> > > > > -1 to removing HTrace from HBase.
> > > > >
> > > > > Upgrade to the desired version, that would be fine.
> > > > >
> > > > >
> > > > ​t​
> > > >
> > > > What you thinking Andrew?
> > > >
> > > > Trace is broke whether we upgrade or not. As is, we mislead. We give
> > off
> > > > the impression that hooking up hbase and hdfs via trace tells a story
> > > about
> > > > request traversals when it at best you'd get a wonky view.
> > > >
> > > > HTrace needs work. HBase and HDFS also need work before we can
> suggest
> > to
> > > > our users this system is viable debugging.
> > > >
> > > > What you reckon Andrew?
> > > > Thanks,
> > > > S
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > On Tue, Aug 15, 2017 at 5:54 AM, Tamas Penzes  >
> > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > As far as I know the actual integration of HTrace 3.2 into HBase
> is
> > > far
> > > > > > from satisfying.
> > > > > >
> > > > > > This version of the library is old, buggy, not really maintained
> > and
> > > > the
> > > > > > migration to version 4.x is quite hard (
> > > > > > https://issues.apache.org/jira/browse/HBASE-14451) and looks
> like
> > > the
> > > > > task
> > > > > > is not getting much closer to the goal.
> > > > > >
> > > > > > This is why I think we should remove HTrace now and if we still
> > want
> > > to
> > > > > > have some art of tracing (I think it would be useful) then start
> it
> > > > from
> > > > > > the baseline.
> > > > > > I have created a ticket for this task:
> > > > > > https://issues.apache.org/jira/browse/HBASE-18601
> > > > > > The removal could be quite fast and reach version 2.0.0 too.
> > > > > >
> > > > > > What do you all think?
> > > > > >
> > > > > > Thanks, Tamaas
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew
> > > > >
> > > > > Words like orphans lost among the crosstalk, meaning torn from
> > truth's
> > > > > decrepit hands
> > > > >- A23, Crosstalk
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >- A23, Crosstalk
> > >
> >
>

[jira] [Created] (HBASE-18610) Provide capability to activate chaos monkey

[Ted Yu (JIRA)]

Ted Yu created HBASE-18610:
--

 Summary: Provide capability to activate chaos monkey
 Key: HBASE-18610
 URL: https://issues.apache.org/jira/browse/HBASE-18610
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu


Currently load-client runs against a cluster where region servers are stable.

We need to introduce chaos monkey so that wider coverage for read path is 
exercised.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18609) Apply HBASE-15511 in code base

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-18609:
--

 Summary: Apply HBASE-15511 in code base
 Key: HBASE-18609
 URL: https://issues.apache.org/jira/browse/HBASE-18609
 Project: HBase
  Issue Type: Improvement
Reporter: Chia-Ping Tsai


HBASE-15511 enable us to get the cluster status by scope. We should apply it in 
code base so as to prevent the useless information.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18608) AsyncConnection should return AsyncAdmin interface instead of the implemenation

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-18608:
--

 Summary: AsyncConnection should return AsyncAdmin interface 
instead of the implemenation
 Key: HBASE-18608
 URL: https://issues.apache.org/jira/browse/HBASE-18608
 Project: HBase
  Issue Type: Sub-task
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang


hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnection.java
{code}
AsyncAdminBuilder getAdminBuilder();
AsyncAdminBuilder getAdminBuilder(ExecutorService pool);
{code}
These two method should not use the implemenation: RawAsyncHBaseAdmin and 
AsyncHBaseAdmin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18607) fix submit-patch.py to support utf8

[Tamas Penzes (JIRA)]

Tamas Penzes created HBASE-18607:


 Summary: fix submit-patch.py to support utf8
 Key: HBASE-18607
 URL: https://issues.apache.org/jira/browse/HBASE-18607
 Project: HBase
  Issue Type: Bug
Reporter: Tamas Penzes
Assignee: Tamas Penzes


If we have a change in files which contain UTF-8 characters the script is not 
able to submit the patch. Since the License.vm file contains such characters we 
have to support it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)