[jira] [Commented] (HBASE-19290) Reduce zk request when doing split log
[
https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256632#comment-16256632
]
Appy commented on HBASE-19290:
--
- Please add a short commit message explaining 'idea' of the change (makes
reviewing easy then having to reverse engineer the 'idea')
- grabTask only returns 0 or 1. Use boolean please.
- Add @return comment to grabTask.
- We probably don't need {{if (taskGrabed == 0 && !shouldStop) }},
taskReadySeq.wait is indefinite wait on availability of next task.
- Can we pull the while loop one level outside and share the
{{if(calculateAvailableSplitters(..))}} condition.
> Reduce zk request when doing split log
> --
>
> Key: HBASE-19290
> URL: https://issues.apache.org/jira/browse/HBASE-19290
> Project: HBase
> Issue Type: Improvement
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-19290.master.001.patch
>
>
> We observe once the cluster has 1000+ nodes and when hundreds of nodes abort
> and doing split log, the split is very very slow, and we find the
> regionserver and master wait on the zookeeper response, so we need to reduce
> zookeeper request and pressure for big cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Attachment: HBASE-18309.master.008.patch Update as [~aoxiang] reviews. > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Status: Patch Available (was: Open) > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19192) HMaster#getRegionServerInfoPort() should respect config value of -1
[
https://issues.apache.org/jira/browse/HBASE-19192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256808#comment-16256808
]
Janos Gub commented on HBASE-19192:
---
A quick question: Isn't regionServerInfo.getInfoPort coming from the
Regionserver's set up REGRIONSERVER_INFO_PORT?
In HRegionServer.java:
{code:java}
private void createMyEphemeralNode() throws KeeperException, IOException {
RegionServerInfo.Builder rsInfo = RegionServerInfo.newBuilder();
rsInfo.setInfoPort(infoServer != null ? infoServer.getPort() : -1);
rsInfo.setVersionInfo(ProtobufUtil.getVersionInfo());
byte[] data = ProtobufUtil.prependPBMagic(rsInfo.build().toByteArray());
ZKUtil.createEphemeralNodeAndWatch(this.zooKeeper,
getMyEphemeralNodePath(), data);
}
{code}
I think the ephemeral node contains the correct info port for a regionserver.
The code in the description fetches this information. In this case this is not
an error.
[~tedyu] Could you please confirm?
> HMaster#getRegionServerInfoPort() should respect config value of -1
> ---
>
> Key: HBASE-19192
> URL: https://issues.apache.org/jira/browse/HBASE-19192
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> public int getRegionServerInfoPort(final ServerName sn) {
> RegionServerInfo info = this.regionServerTracker.getRegionServerInfo(sn);
> if (info == null || info.getInfoPort() == 0) {
> return conf.getInt(HConstants.REGIONSERVER_INFO_PORT,
> HConstants.DEFAULT_REGIONSERVER_INFOPORT);
> }
> return info.getInfoPort();
> {code}
> hbase.regionserver.info.port config is only checked when regionServerTracker
> doesn't have info port.
> When hbase.regionserver.info.port is set to -1 by user, we should respect the
> config value and disable UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Status: Patch Available (was: Open) > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > HBASE-18309.master.009.patch, space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256704#comment-16256704
]
Samarth Jain commented on HBASE-18359:
--
We have been cloning config to make sure the changes we are making do not
introduce side effects on other regions of the region server. Something like
this:
{code}
/*
* We need to create a copy of region's configuration since we don't want any
side effect of
* setting the RpcControllerFactory.
*/
clonedConfig = PropertiesUtil.cloneConfig(e.getConfiguration());
{code}
On this clonedConfig we also set the various timeout and retry related configs
that [~jamestaylor] mentioned. This clonedConfig is then to our
CoprocessorHConnectionTableFactory that makes sure that the HConnection used by
the HTables generated from this factory use this cloned config. The HConnection
is today created by doing this:
{code}
new CoprocessorHConnection(clonedConfig, server)
{code}
So we would need a way of creating this "short circuit optimized" HConnection
by having it take a configuration object.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19273) IntegrationTestBulkLoad#installSlowingCoproc() uses read-only HTableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-19273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19273:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-beta-1
Status: Resolved (was: Patch Available)
Thanks for the reviews.
> IntegrationTestBulkLoad#installSlowingCoproc() uses read-only HTableDescriptor
> --
>
> Key: HBASE-19273
> URL: https://issues.apache.org/jira/browse/HBASE-19273
> Project: HBase
> Issue Type: Test
>Reporter: Romil Choksi
>Assignee: Ted Yu
> Fix For: 2.0.0-beta-1
>
> Attachments: 19273.v1.txt, 19273.v2.txt
>
>
> [~romil.choksi] reported the following :
> {code}
> 2017-11-15 23:03:04,455 ERROR [main] util.AbstractHBaseTool: Error running
> command-line tool
> java.lang.UnsupportedOperationException: HTableDescriptor is read-only
> at
> org.apache.hadoop.hbase.client.ImmutableHTableDescriptor.getDelegateeForModification(ImmutableHTableDescriptor.java:59)
> at
> org.apache.hadoop.hbase.HTableDescriptor.addCoprocessor(HTableDescriptor.java:710)
> at
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.installSlowingCoproc(IntegrationTestBulkLoad.java:215)
> at
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.testBulkLoad(IntegrationTestBulkLoad.java:222)
> at
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.runTestFromCommandLine(IntegrationTestBulkLoad.java:790)
> at
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:155)
> {code}
> This is due to read only descriptor being used.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18946) Stochastic load balancer assigns replica regions to the same RS
[ https://issues.apache.org/jira/browse/HBASE-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256922#comment-16256922 ] ramkrishna.s.vasudevan commented on HBASE-18946: Some updates here -> In roundRobinAssignment the LB does not even care for the replicas and its assignment. So we need to make it aware of it. It knows that the region has a replica but it does not check if same replica is being assigned only balancer knows about it. -> Say we want to assign replica 3 then we have to ensure that the primary and the secondary replica are already assigned and from that determine a plan. So to do that we may have to enable the cost functions or we should add some new methods to make LB in roundrobin to be aware of replicas. -> Next is that the way we do in the current patch attached here actually solved the issue because for the balancer it got all the regions (incluidng replicas) and just applied the round robin part on it. Will be back here. > Stochastic load balancer assigns replica regions to the same RS > --- > > Key: HBASE-18946 > URL: https://issues.apache.org/jira/browse/HBASE-18946 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18946.patch, HBASE-18946.patch, > TestRegionReplicasWithRestartScenarios.java > > > Trying out region replica and its assignment I can see that some times the > default LB Stocahstic load balancer assigns replica regions to the same RS. > This happens when we have 3 RS checked in and we have a table with 3 > replicas. When a RS goes down then the replicas being assigned to same RS is > acceptable but the case when we have enough RS to assign this behaviour is > undesirable and does not solve the purpose of replicas. > [~huaxiang] and [~enis]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256705#comment-16256705 ] ramkrishna.s.vasudevan commented on HBASE-19092: The main discussion with Anoop was that if to really add the new Tag APIs in ExtendedCell (though we make it LP). So he was of the opinion that since CPs are not using ExtendedCell if we add these APIs in ExtendedCell it would mean that we need to do a typecast and that is not straightforward. Instead have some static methods only. So shall we add getTags(Cell) to the Tag interface only? Currently Tag is only for specific Tags but lets change it now? If not the other options is introduce 'Tags' (note the 's') interface and add getTags(Cell) to that interface and also have 'Tag' interface. Thoughts/suggstion? [~saint@gmail.com] and [~chia7712]. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256726#comment-16256726
]
Anoop Sam John commented on HBASE-18359:
This issue (As stated in subject) is no longer valid for latest HBase. On
older versions also it is not as the
CoprocessorHConnection#getConnectionForEnvironment was taking a
CoprocessorEnvironment instance not config. So to change the configs, u will
end up changing the actual region level config. Ya to solve the issue (said in
this jira) having a workaround of CoprocessorHConnectionTableFactory. Ya what
that doing is correct. From 2.0 u can not have this
CoprocessorHConnectionTableFactory. What we need in HBase is an API
CoprocessorEnvironment#getConnection(Config). This call will always make new
connection (which is short circuit enabled). So caching of this and reuse the
callee has to take care. Is that ok? Then we can provide a quick patch and
make it avail in beta-1 release. I will close down this jira and open a new
one with correct title and desc.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19291) Use common header and footer for JSP pages
[
https://issues.apache.org/jira/browse/HBASE-19291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256831#comment-16256831
]
Hadoop QA commented on HBASE-19291:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}164m
54s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898142/HBASE-19291.master.001.patch
|
| Optional Tests | asflicense xml javac javadoc unit |
| uname | Linux e6e738078680 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e6e731cb86 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9887/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9887/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Use common header and footer for JSP pages
> --
>
> Key: HBASE-19291
> URL: https://issues.apache.org/jira/browse/HBASE-19291
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-19291.master.001.patch
>
>
> Use header and footer in our *.jsp pages to avoid unnecessary redundancy
> (copy-paste of code)
> (Been sitting in my local repo for long, best to get following pesky
> user-facing things fixed before the next major release)
> Misc edits:
> - Due to redundancy, new additions make it to some places but not others. For
> eg there are missing links to "/logLevel", "/processRS.jsp" in few places.
> - Fix processMaster.jsp wrongly pointing to rs-status instead of
> master-status (probably due to copy paste from processRS.jsp)
> - Deleted a bunch of extraneous "" in processMaster.jsp & processRS.jsp
> - Added missing tag in snapshot.jsp
> - Deleted fossils of html5shiv.js. It's uses and the js itself were deleted
> in the commit "819aed4ccd073d818bfef5931ec8d248bfae5f1f"
> - Fixed wrongly matched heading tags
> - Deleted some unused variables
> Tested:
> Ran standalone cluster and opened each
[jira] [Updated] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19274:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Guangxu.
> Log IOException when unable to determine the size of committed file
> ---
>
> Key: HBASE-19274
> URL: https://issues.apache.org/jira/browse/HBASE-19274
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
>Priority: Trivial
> Fix For: 2.0
>
> Attachments: HBASE-19274.master.001.patch
>
>
> During troubleshooting of slow response, I saw the following in region server
> log:
> {code}
> 2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed to find the size of hfile
> hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
> {code}
> Here is related code:
> {code}
> } catch (IOException e) {
> LOG.warn("Failed to find the size of hfile " +
> commitedStoreFile);
> {code}
> The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Attachment: HBASE-18309.master.009.patch Restore interrupt status when interruption occurs while cleaning oldWALs. > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > HBASE-18309.master.009.patch, space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256652#comment-16256652
]
James Taylor commented on HBASE-18359:
--
The other reason we override the coprocessor environment is to set the timeout
and retries to a lower value so that RS->RS calls fail fast (rather than the
opposite - doing 10x retries than what's configured). This is important so that
we don't tie up a handler thread for a long period of time.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19148) Edit of default configuration
[
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256667#comment-16256667
]
Ashish Singhi commented on HBASE-19148:
---
The default value 0.1f for {{replication.source.ratio}} is also very less, we
should increase it, there is a jira also to handle that HBASE-16499
> Edit of default configuration
> -
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
> Issue Type: Bug
> Components: defaults
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change
> defaults given experience.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256688#comment-16256688
]
Anoop Sam John commented on HBASE-18359:
Ya that make sense James. So what u say is u can not make this config changes
globally in xml in the RS side . Correct? With current 2.0 you can not do
this. Ya u can create connection how u make at client side. But will not get
the short circuit benefit then. So if u need this, we will have to add new way
then. Let us know.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18967: -- Status: Patch Available (was: In Progress) > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18967: -- Status: In Progress (was: Patch Available) > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Status: Open (was: Patch Available) > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256834#comment-16256834
]
Ted Yu commented on HBASE-19274:
[ERROR]
TestStochasticLoadBalancer2.testRegionReplicasOnMidClusterHighReplication:87->BalancerTestBase.testWithCluster:525->BalancerTestBase.testWithCluster:553->BalancerTestBase.assertRegionReplicaPlacement:263
Two or more region replicas are hosted on the same host after balance
Not related to patch.
+1
> Log IOException when unable to determine the size of committed file
> ---
>
> Key: HBASE-19274
> URL: https://issues.apache.org/jira/browse/HBASE-19274
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
>Priority: Trivial
> Attachments: HBASE-19274.master.001.patch
>
>
> During troubleshooting of slow response, I saw the following in region server
> log:
> {code}
> 2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed to find the size of hfile
> hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
> {code}
> Here is related code:
> {code}
> } catch (IOException e) {
> LOG.warn("Failed to find the size of hfile " +
> commitedStoreFile);
> {code}
> The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19123) Purge 'complete' support from Coprocesor Observers
[
https://issues.apache.org/jira/browse/HBASE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256736#comment-16256736
]
Hadoop QA commented on HBASE-19123:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
1s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} hbase-server: The patch generated 0 new + 50
unchanged - 3 fixed = 50 total (was 53) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
52m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 17s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHStore |
| | hadoop.hbase.security.token.TestZKSecretWatcher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19123 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898138/HBASE-19123.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fafd2df3ee0d 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / e6e731cb86 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9886/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9886/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-19288) Intermittent test failure in TestHStore.testRunDoubleMemStoreCompactors
[
https://issues.apache.org/jira/browse/HBASE-19288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256827#comment-16256827
]
Ted Yu commented on HBASE-19288:
[~anoop.hbase] [~ramkrishna.s.vasude...@gmail.com]:
Can you take a look ?
> Intermittent test failure in TestHStore.testRunDoubleMemStoreCompactors
> ---
>
> Key: HBASE-19288
> URL: https://issues.apache.org/jira/browse/HBASE-19288
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19288.v1.txt, testRunDoubleMemStoreCompactors.out
>
>
> Here was one of the test failures:
> https://builds.apache.org/job/PreCommit-HBASE-Build/9812/testReport/junit/org.apache.hadoop.hbase.regionserver/TestHStore/testRunDoubleMemStoreCompactors/
>
> {code}
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestHStore.testRunDoubleMemStoreCompactors(org.apache.hadoop.hbase.regionserver.TestHStore)
> [ERROR] Run 1: TestHStore.testRunDoubleMemStoreCompactors:1500 expected:<2>
> but was:<3>
> [ERROR] Run 2: TestHStore.testRunDoubleMemStoreCompactors:1481 expected:<1>
> but was:<4>
> [ERROR] Run 3: TestHStore.testRunDoubleMemStoreCompactors:1481 expected:<1>
> but was:<5>
> {code}
> From the counts for second and third runs, we know that RUNNER_COUNT was not
> cleared in between the reruns, leading to failure at the 1st assertion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19274:
---
Fix Version/s: (was: 2.0)
2.0.0-beta-1
> Log IOException when unable to determine the size of committed file
> ---
>
> Key: HBASE-19274
> URL: https://issues.apache.org/jira/browse/HBASE-19274
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
>Priority: Trivial
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19274.master.001.patch
>
>
> During troubleshooting of slow response, I saw the following in region server
> log:
> {code}
> 2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed to find the size of hfile
> hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
> {code}
> Here is related code:
> {code}
> } catch (IOException e) {
> LOG.warn("Failed to find the size of hfile " +
> commitedStoreFile);
> {code}
> The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256641#comment-16256641
]
Hadoop QA commented on HBASE-19274:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 5m
2s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 8m
44s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
89m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}139m 15s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}263m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19274 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898123/HBASE-19274.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f03735f22074 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 52273aa8f3 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9884/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9884/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9884/console |
| Powered by | Apache Yetus 0.6.0
[jira] [Updated] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18967: -- Attachment: HBASE-18967.branch-1.3.003.patch > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18309: -- Status: Open (was: Patch Available) > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19293) Support add a disabled state replication peer directly
[ https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19293: --- Attachment: HBASE-19293.master.001.patch > Support add a disabled state replication peer directly > -- > > Key: HBASE-19293 > URL: https://issues.apache.org/jira/browse/HBASE-19293 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-19293.master.001.patch > > > Now when add a replication peer, the default state is enabled. If you want > add a disabled replication peer, you need add a peer first, then disable it. > It need two step to finish now. > Use case for add a disabled replication peer. When user want sync data from a > cluster A to a new peer cluster. > 1. Add a disabled replication peer. And config the table to peer config. > 2. Take a snapshot of table and export snapshot to peer cluster. > 3. Restore snapshot in peer cluster. > 4. Enable the peer and wait all stuck replication log replicated to peer > cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19293) Support add a disabled state replication peer directly
[ https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19293: --- Fix Version/s: 2.0.0-beta-1 > Support add a disabled state replication peer directly > -- > > Key: HBASE-19293 > URL: https://issues.apache.org/jira/browse/HBASE-19293 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19293.master.001.patch > > > Now when add a replication peer, the default state is enabled. If you want > add a disabled replication peer, you need add a peer first, then disable it. > It need two step to finish now. > Use case for add a disabled replication peer. When user want sync data from a > cluster A to a new peer cluster. > 1. Add a disabled replication peer. And config the table to peer config. > 2. Take a snapshot of table and export snapshot to peer cluster. > 3. Restore snapshot in peer cluster. > 4. Enable the peer and wait all stuck replication log replicated to peer > cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19293) Support add a disabled state replication peer directly
[ https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19293: --- Status: Patch Available (was: Open) > Support add a disabled state replication peer directly > -- > > Key: HBASE-19293 > URL: https://issues.apache.org/jira/browse/HBASE-19293 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-19293.master.001.patch > > > Now when add a replication peer, the default state is enabled. If you want > add a disabled replication peer, you need add a peer first, then disable it. > It need two step to finish now. > Use case for add a disabled replication peer. When user want sync data from a > cluster A to a new peer cluster. > 1. Add a disabled replication peer. And config the table to peer config. > 2. Take a snapshot of table and export snapshot to peer cluster. > 3. Restore snapshot in peer cluster. > 4. Enable the peer and wait all stuck replication log replicated to peer > cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18654) HBaseTestingUtility - sessions still active after cluster shutdown
[
https://issues.apache.org/jira/browse/HBASE-18654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256891#comment-16256891
]
Evgenii Ganchurin commented on HBASE-18654:
---
I reproduced the issue on 1.2.5. As mentioned in the description, the issue
occurs on attempt to shutdown HBaseTestingUtility. Additionally, it blocks
running one more instance of HBaseTestingUtility. Everything you can find in
logs are instances of ConnectException thrown in infinite loop.
[~jamesnetherton] Were you able to solve the issue somehow?
> HBaseTestingUtility - sessions still active after cluster shutdown
> --
>
> Key: HBASE-18654
> URL: https://issues.apache.org/jira/browse/HBASE-18654
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6
>Reporter: James Netherton
>
> After calling {{shutdownMiniCluster()}} I see my logs littered with entries
> like:
> {code}
> 2017-08-22 20:33:18 WARN [org.apache.zookeeper.ClientCnxn]
> (localhost:41049.activeMasterManager-SendThread(localhost:55128)) - Session
> 0x15e0b6ef4fe0005 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> {code}
> I tried various versions of HBase and Zookeeper but I hit this problem
> regardless.
> You can replicate easily with a test that does:
> {code}
> HBaseTestingUtility utility = new HBaseTestingUtility();
> utility.startMiniCluster(1);
> utility.shutdownMiniCluster();
> Thread.sleep(3);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18967: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Push to branch-1.3. Thanks for the patch. [~psomogyi] > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257056#comment-16257056 ] Hudson commented on HBASE-18967: FAILURE: Integrated in Jenkins build HBase-1.3-IT #290 (See [https://builds.apache.org/job/HBase-1.3-IT/290/]) HBASE-18967 Backport HBASE-17181 to branch-1.3 (Let HBase thrift2 (chia7712: rev 746447cf6487da3f0c9ec72b3824e9c99e88430e) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257057#comment-16257057 ] Hudson commented on HBASE-17181: FAILURE: Integrated in Jenkins build HBase-1.3-IT #290 (See [https://builds.apache.org/job/HBase-1.3-IT/290/]) HBASE-18967 Backport HBASE-17181 to branch-1.3 (Let HBase thrift2 (chia7712: rev 746447cf6487da3f0c9ec72b3824e9c99e88430e) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 1.4.0 >Reporter: Jian Yi >Assignee: Jian Yi >Priority: Minor > Labels: features > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch, HBASE-17181-V4.patch, HBASE-17181-V5.patch, > HBASE-17181-V6.patch, ThriftServer.java > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19294) When user trying to create table , master is getting aborted in execution phase
chiranjeevi created HBASE-19294: --- Summary: When user trying to create table , master is getting aborted in execution phase Key: HBASE-19294 URL: https://issues.apache.org/jira/browse/HBASE-19294 Project: HBase Issue Type: Bug Components: hbase, master, proc-v2 Affects Versions: 1.3.1 Reporter: chiranjeevi when create table failed in execute phase master got aborted. DEBUG [RpcServer.FifoWFPBQ.default.handler=8,queue=3,port=16000] ipc.RpcServer: RpcServer.FifoWFPBQ.default.handler=8,queue=3,port=16000: callId: 443 service: MasterService methodName: ListTableDescriptorsByNamespace size: 51 connection: java.io.InterruptedIOException: Retry interrupted java.lang.RuntimeException: HMaster Aborted at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:239) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:137) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3207) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI
[ https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257098#comment-16257098 ] Ted Yu commented on HBASE-17436: [~appy]: Can you take a look ? > Add facility to provide more information for Other Regions seen on Master UI > > > Key: HBASE-17436 > URL: https://issues.apache.org/jira/browse/HBASE-17436 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Janos Gub > Labels: ui, usability > Fix For: 2.0.0 > > Attachments: 17436.v10.txt, 17436.v9.txt, HBASE-17436-v2.patch, > HBASE-17436-v5.patch, HBASE-17436-v6.patch, HBASE-17436-v6.patch, > HBASE-17436-v7.patch, HBASE-17436-v8.patch, HBASE-17436.patch, > HBASE-17779-v3.patch, HBASE-17779-v4.patch, Screen Shot 2017-04-24 at 8.43.31 > PM.png, Screen Shot 2017-04-26 at 4.39.49 PM.png, Screen Shot 2017-11-08 at > 10.29.49 AM.png, Screen Shot 2017-11-08 at 10.30.09 AM.png, initial.patch, > regionstates.png, table-selected.png, tableregions.png > > > [~rpednekar] and I were looking at a case where the count displayed under > Other Regions was high (~1200). > Since the table page just maintains a count instead of List of region names, > it is very difficult for user to determine which regions belong to this > category. > We should provide facility to provide more details for this category (LOG, > JMX, etc). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257048#comment-16257048
]
Hudson commented on HBASE-19274:
FAILURE: Integrated in Jenkins build HBase-2.0 #867 (See
[https://builds.apache.org/job/HBase-2.0/867/])
HBASE-19274 Log IOException when unable to determine the size of (tedyu: rev
63b02a072a5057e4da2645611c0e5d7f13132088)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> Log IOException when unable to determine the size of committed file
> ---
>
> Key: HBASE-19274
> URL: https://issues.apache.org/jira/browse/HBASE-19274
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
>Priority: Trivial
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19274.master.001.patch
>
>
> During troubleshooting of slow response, I saw the following in region server
> log:
> {code}
> 2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed to find the size of hfile
> hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
> {code}
> Here is related code:
> {code}
> } catch (IOException e) {
> LOG.warn("Failed to find the size of hfile " +
> commitedStoreFile);
> {code}
> The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18938) Backport HBASE-16985 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257053#comment-16257053 ] Peter Somogyi commented on HBASE-18938: --- TestRateLimiter passes me locally. +1 after checkstyle fix. > Backport HBASE-16985 to branch-1.3 > -- > > Key: HBASE-18938 > URL: https://issues.apache.org/jira/browse/HBASE-18938 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18938-branch-1.3.patch, > HBASE-18938-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18937) Backport HBASE-16815 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18937: -- Attachment: HBASE-18937-branch-1.3.patch > Backport HBASE-16815 to branch-1.3 > -- > > Key: HBASE-18937 > URL: https://issues.apache.org/jira/browse/HBASE-18937 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18937-branch-1.3.patch, > HBASE-18937-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19159) Backup should check permission for snapshot copy in advance
[
https://issues.apache.org/jira/browse/HBASE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257068#comment-16257068
]
Janos Gub commented on HBASE-19159:
---
I would like to work on this issue. Attaching an initial patch, still trying to
write a test for it.
> Backup should check permission for snapshot copy in advance
> ---
>
> Key: HBASE-19159
> URL: https://issues.apache.org/jira/browse/HBASE-19159
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Minor
>
> When the user running the backup doesn't have permission to copy the snapshot
> , he / she would see:
> {code}
> 2017-11-02 18:21:33,654 ERROR [main] util.AbstractHBaseTool: Error running
> command-line tool
> org.apache.hadoop.hbase.snapshot.ExportSnapshotException: Failed to copy the
> snapshot directory:
> from=hdfs://ctr-e134-1499953498516-263664-01-03.hwx.site:8020/apps/hbase/data/.hbase-snapshot/snapshot_1509646891251_default_IntegrationTestBackupRestore.table2
>
> to=hdfs://ctr-e134-1499953498516-263664-01-03.hwx.site:8020/user/root/test-data/fb919a6f-3cb4-4d57-bbcf-561d6e5b3ae8/backupIT/backup_1509646884252/default/IntegrationTestBackupRestore.table2/.hbase-snapshot/.tmp/snapshot_1509646891251_default_IntegrationTestBackupRestore.table2
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.doWork(ExportSnapshot.java:1009)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
> at
> org.apache.hadoop.hbase.backup.mapreduce.MapReduceBackupCopyJob.copy(MapReduceBackupCopyJob.java:386)
> at
> org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.snapshotCopy(FullTableBackupClient.java:103)
> at
> org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.execute(FullTableBackupClient.java:175)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.runTest(IntegrationTestBackupRestore.java:180)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.testBackupRestore(IntegrationTestBackupRestore.java:134)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.runTestFromCommandLine(IntegrationTestBackupRestore.java:263)
> {code}
> It would be more user friendly if the permission is checked before taking the
> snapshot.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19159) Backup should check permission for snapshot copy in advance
[
https://issues.apache.org/jira/browse/HBASE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Gub updated HBASE-19159:
--
Attachment: initial_patch.txt
> Backup should check permission for snapshot copy in advance
> ---
>
> Key: HBASE-19159
> URL: https://issues.apache.org/jira/browse/HBASE-19159
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Minor
> Attachments: initial_patch.txt
>
>
> When the user running the backup doesn't have permission to copy the snapshot
> , he / she would see:
> {code}
> 2017-11-02 18:21:33,654 ERROR [main] util.AbstractHBaseTool: Error running
> command-line tool
> org.apache.hadoop.hbase.snapshot.ExportSnapshotException: Failed to copy the
> snapshot directory:
> from=hdfs://ctr-e134-1499953498516-263664-01-03.hwx.site:8020/apps/hbase/data/.hbase-snapshot/snapshot_1509646891251_default_IntegrationTestBackupRestore.table2
>
> to=hdfs://ctr-e134-1499953498516-263664-01-03.hwx.site:8020/user/root/test-data/fb919a6f-3cb4-4d57-bbcf-561d6e5b3ae8/backupIT/backup_1509646884252/default/IntegrationTestBackupRestore.table2/.hbase-snapshot/.tmp/snapshot_1509646891251_default_IntegrationTestBackupRestore.table2
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.doWork(ExportSnapshot.java:1009)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
> at
> org.apache.hadoop.hbase.backup.mapreduce.MapReduceBackupCopyJob.copy(MapReduceBackupCopyJob.java:386)
> at
> org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.snapshotCopy(FullTableBackupClient.java:103)
> at
> org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.execute(FullTableBackupClient.java:175)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.runTest(IntegrationTestBackupRestore.java:180)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.testBackupRestore(IntegrationTestBackupRestore.java:134)
> at
> org.apache.hadoop.hbase.IntegrationTestBackupRestore.runTestFromCommandLine(IntegrationTestBackupRestore.java:263)
> {code}
> It would be more user friendly if the permission is checked before taking the
> snapshot.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[
https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256976#comment-16256976
]
Chia-Ping Tsai commented on HBASE-19092:
bq. So he was of the opinion that since CPs are not using ExtendedCell if we
add these APIs in ExtendedCell it would mean that we need to do a typecast and
that is not straightforward.
Indeed, it is not straightforward but it is the best solution now I think.
bq. Instead have some static methods only. So shall we add getTags(Cell) to the
Tag interface only? Currently Tag is only for specific Tags but lets change it
now?
The {{getTags(Cell)}} is in {{PrivateCellUtil}}. Do you mean moving the method
to {{Tag}}? Typecasting the {{Cell}} to {{ExtendedCell}} is enough for cp user
if they want to get {{Tag}} from {{Cell}}.
> Make Tag IA.LimitedPrivate and expose for CPs
> -
>
> Key: HBASE-19092
> URL: https://issues.apache.org/jira/browse/HBASE-19092
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19092-branch-2.patch,
> HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch,
> HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch,
> HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch
>
>
> We need to make tags as LimitedPrivate as some use cases are trying to use
> tags like timeline server. The same topic was discussed in dev@ and also in
> HBASE-18995.
> Shall we target this for beta1 - cc [~saint@gmail.com].
> So once we do this all related Util methods and APIs should also move to
> LimitedPrivate Util classes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257040#comment-16257040 ] Hudson commented on HBASE-18967: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #370 (See [https://builds.apache.org/job/HBase-1.3-JDK8/370/]) HBASE-18967 Backport HBASE-17181 to branch-1.3 (Let HBase thrift2 (chia7712: rev 746447cf6487da3f0c9ec72b3824e9c99e88430e) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257041#comment-16257041 ] Hudson commented on HBASE-17181: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #370 (See [https://builds.apache.org/job/HBase-1.3-JDK8/370/]) HBASE-18967 Backport HBASE-17181 to branch-1.3 (Let HBase thrift2 (chia7712: rev 746447cf6487da3f0c9ec72b3824e9c99e88430e) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 1.4.0 >Reporter: Jian Yi >Assignee: Jian Yi >Priority: Minor > Labels: features > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch, HBASE-17181-V4.patch, HBASE-17181-V5.patch, > HBASE-17181-V6.patch, ThriftServer.java > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257158#comment-16257158
]
Ted Yu commented on HBASE-19289:
[~busbey]:
To unblock testing against hadoop3, how about relaxing the following for unit
tests ?
{code}
if (!(CommonFSUtils.hasCapability(output, "hflush"))) {
throw new StreamLacksCapabilityException("hflush");
{code}
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257013#comment-16257013 ] Chia-Ping Tsai commented on HBASE-18967: Will commit it later. > Backport HBASE-17181 to branch-1.3 > -- > > Key: HBASE-18967 > URL: https://issues.apache.org/jira/browse/HBASE-18967 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch, > HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch, > HBASE-18967.branch-1.3.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18309) Support multi threads in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257029#comment-16257029
]
Hadoop QA commented on HBASE-18309:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} hbase-server: The patch generated 0 new + 4
unchanged - 4 fixed = 4 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
55m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 98m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18309 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898190/HBASE-18309.master.009.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2dfd67b42cf2 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e8ae6d4b73 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9891/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9891/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Support multi threads in CleanerChore
> -
>
> Key:
[jira] [Assigned] (HBASE-18937) Backport HBASE-16815 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-18937: - Assignee: Peter Somogyi (was: Ashish Singhi) > Backport HBASE-16815 to branch-1.3 > -- > > Key: HBASE-18937 > URL: https://issues.apache.org/jira/browse/HBASE-18937 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18937-branch-1.3.patch, > HBASE-18937-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18937) Backport HBASE-16815 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18937: -- Assignee: Ashish Singhi (was: Peter Somogyi) Status: Patch Available (was: In Progress) > Backport HBASE-16815 to branch-1.3 > -- > > Key: HBASE-18937 > URL: https://issues.apache.org/jira/browse/HBASE-18937 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18937-branch-1.3.patch, > HBASE-18937-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18937) Backport HBASE-16815 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257067#comment-16257067 ] Peter Somogyi commented on HBASE-18937: --- Reattached patch. > Backport HBASE-16815 to branch-1.3 > -- > > Key: HBASE-18937 > URL: https://issues.apache.org/jira/browse/HBASE-18937 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18937-branch-1.3.patch, > HBASE-18937-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256961#comment-16256961 ] Duo Zhang commented on HBASE-19271: --- Any concerns? [~appy] [~stack]. Thanks. > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[
https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256976#comment-16256976
]
Chia-Ping Tsai edited comment on HBASE-19092 at 11/17/17 2:04 PM:
--
bq. So he was of the opinion that since CPs are not using ExtendedCell if we
add these APIs in ExtendedCell it would mean that we need to do a typecast and
that is not straightforward.
Indeed, it is not straightforward but it is the best solution now I think.
bq. Instead have some static methods only. So shall we add getTags(Cell) to the
Tag interface only? Currently Tag is only for specific Tags but lets change it
now?
The {{getTags(Cell)}} is in {{PrivateCellUtil}}. Do you mean moving the method
to {{Tag}}? Typecasting the {{Cell}} to {{ExtendedCell}} is good enough for cp
user if they want to get {{Tag}} from {{Cell}}.
was (Author: chia7712):
bq. So he was of the opinion that since CPs are not using ExtendedCell if we
add these APIs in ExtendedCell it would mean that we need to do a typecast and
that is not straightforward.
Indeed, it is not straightforward but it is the best solution now I think.
bq. Instead have some static methods only. So shall we add getTags(Cell) to the
Tag interface only? Currently Tag is only for specific Tags but lets change it
now?
The {{getTags(Cell)}} is in {{PrivateCellUtil}}. Do you mean moving the method
to {{Tag}}? Typecasting the {{Cell}} to {{ExtendedCell}} is enough for cp user
if they want to get {{Tag}} from {{Cell}}.
> Make Tag IA.LimitedPrivate and expose for CPs
> -
>
> Key: HBASE-19092
> URL: https://issues.apache.org/jira/browse/HBASE-19092
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19092-branch-2.patch,
> HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch,
> HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch,
> HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch
>
>
> We need to make tags as LimitedPrivate as some use cases are trying to use
> tags like timeline server. The same topic was discussed in dev@ and also in
> HBASE-18995.
> Shall we target this for beta1 - cc [~saint@gmail.com].
> So once we do this all related Util methods and APIs should also move to
> LimitedPrivate Util classes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18967) Backport HBASE-17181 to branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257010#comment-16257010
]
Chia-Ping Tsai commented on HBASE-18967:
{code}
22:10:49 | Vote | Subsystem | Runtime | Comment
22:10:49
22:10:49 | | || Prechecks
22:10:49 | +1 | hbaseanti | 0m 0s | Patch does not have any
anti-patterns.
22:10:49 | +1 |@author | 0m 0s | The patch does not contain any
@author
22:10:49 | | || tags.
22:10:49 | -1 | test4tests | 0m 0s | The patch doesn't appear to
include any
22:10:49 | | || new or modified tests. Please
justify
22:10:49 | | || why no new tests are needed
for this
22:10:49 | | || patch. Also please list what
manual
22:10:49 | | || steps were performed to verify
this
22:10:49 | | || patch.
22:10:49 | | || branch-1.3 Compile Tests
22:10:49 | +1 | mvninstall | 1m 0s | branch-1.3 passed
22:10:49 | +1 |compile | 0m 11s | branch-1.3 passed
22:10:49 | +1 | checkstyle | 0m 36s | branch-1.3 passed
22:10:49 | +1 | findbugs | 0m 32s | branch-1.3 passed
22:10:49 | +1 |javadoc | 0m 45s | branch-1.3 passed
22:10:49 | | || Patch Compile Tests
22:10:49 | +1 | mvninstall | 0m 11s | the patch passed
22:10:49 | +1 |compile | 0m 10s | the patch passed
22:10:49 | +1 | javac | 0m 10s | the patch passed
22:10:49 | +1 | checkstyle | 0m 13s | hbase-thrift: The patch
generated 0 new
22:10:49 | | || + 8 unchanged - 5 fixed = 8
total (was
22:10:49 | | || 13)
22:10:49 | +1 | whitespace | 0m 0s | The patch has no whitespace
issues.
22:10:49 | +1 |hadoopcheck | 9m 14s | The patch does not cause any
errors
22:10:49 | | || with Hadoop 2.4.0 2.4.1 2.5.0
2.5.1
22:10:49 | | || 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1.
22:10:49 | +1 | findbugs | 0m 34s | the patch passed
22:10:49 | +1 |javadoc | 0m 10s | the patch passed
22:10:49 | | || Other Tests
22:10:49 | +1 | unit | 4m 59s | hbase-thrift in the patch
passed.
22:10:49 | +1 | asflicense | 0m 31s | The patch does not generate
ASF License
22:10:49 | | || warnings.
22:10:49 | | | 19m 37s |
{code}
+1 on v3
> Backport HBASE-17181 to branch-1.3
> --
>
> Key: HBASE-18967
> URL: https://issues.apache.org/jira/browse/HBASE-18967
> Project: HBase
> Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Peter Somogyi
> Fix For: 1.3.2
>
> Attachments: HBASE-18967.branch-1.3.001.patch,
> HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch,
> HBASE-18967.branch-1.3.001.patch, HBASE-18967.branch-1.3.001.patch,
> HBASE-18967.branch-1.3.002.patch, HBASE-18967.branch-1.3.003.patch,
> HBASE-18967.branch-1.3.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18309) Support multi threads in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257047#comment-16257047
]
Hadoop QA commented on HBASE-18309:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 9m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} hbase-server: The patch generated 0 new + 4
unchanged - 4 fixed = 4 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
80m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}141m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}255m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18309 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898176/HBASE-18309.master.008.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e3093e772c8c 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / e6e731cb86 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9889/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9889/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9889/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257058#comment-16257058 ] Reid Chan commented on HBASE-18309: --- Failed tests are not related. > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: binlijin >Assignee: Reid Chan > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > HBASE-18309.master.009.patch, space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19293) Support add a disabled state replication peer directly
[
https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256992#comment-16256992
]
Hadoop QA commented on HBASE-19293:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} hbase-client: The patch generated 1 new + 455
unchanged - 0 fixed = 456 total (was 455) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hbase-replication passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} hbase-server: The patch generated 0 new + 248
unchanged - 3 fixed = 248 total (was 251) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-shell passed checkstyle {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 8 new + 288 unchanged - 7 fixed =
296 total (was 295) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 23 new + 408 unchanged - 1 fixed =
431 total (was 409) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
52m 28s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} |
[jira] [Commented] (HBASE-19293) Support add a disabled state replication peer directly
[ https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256990#comment-16256990 ] Duo Zhang commented on HBASE-19293: --- +1. But let's wait a while for more comments. > Support add a disabled state replication peer directly > -- > > Key: HBASE-19293 > URL: https://issues.apache.org/jira/browse/HBASE-19293 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19293.master.001.patch > > > Now when add a replication peer, the default state is enabled. If you want > add a disabled replication peer, you need add a peer first, then disable it. > It need two step to finish now. > Use case for add a disabled replication peer. When user want sync data from a > cluster A to a new peer cluster. > 1. Add a disabled replication peer. And config the table to peer config. > 2. Take a snapshot of table and export snapshot to peer cluster. > 3. Restore snapshot in peer cluster. > 4. Enable the peer and wait all stuck replication log replicated to peer > cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18937) Backport HBASE-16815 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18937: -- Status: In Progress (was: Patch Available) > Backport HBASE-16815 to branch-1.3 > -- > > Key: HBASE-18937 > URL: https://issues.apache.org/jira/browse/HBASE-18937 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18937-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19274) Log IOException when unable to determine the size of committed file
[
https://issues.apache.org/jira/browse/HBASE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257114#comment-16257114
]
Hudson commented on HBASE-19274:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4068 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4068/])
HBASE-19274 Log IOException when unable to determine the size of (tedyu: rev
e8ae6d4b73191a7ce03c99bccc4eb45b611ce844)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> Log IOException when unable to determine the size of committed file
> ---
>
> Key: HBASE-19274
> URL: https://issues.apache.org/jira/browse/HBASE-19274
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
>Priority: Trivial
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19274.master.001.patch
>
>
> During troubleshooting of slow response, I saw the following in region server
> log:
> {code}
> 2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed to find the size of hfile
> hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
> {code}
> Here is related code:
> {code}
> } catch (IOException e) {
> LOG.warn("Failed to find the size of hfile " +
> commitedStoreFile);
> {code}
> The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19271: - Resolution: Fixed Status: Resolved (was: Patch Available) > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19271: - Fix Version/s: (was: 2.0.0-beta-1) 3.0.0 > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18974) Document "Becoming a Committer"
[ https://issues.apache.org/jira/browse/HBASE-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257425#comment-16257425 ] Misty Stanley-Jones commented on HBASE-18974: - The Hadoop QA error is because my patch needs to be applied on top of yours and then squashed. > Document "Becoming a Committer" > --- > > Key: HBASE-18974 > URL: https://issues.apache.org/jira/browse/HBASE-18974 > Project: HBase > Issue Type: Bug > Components: community, documentation >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18974-copyedit-addendum.patch, HBASE-18974.patch, > HBASE-18974.v2.patch, HBASE-18974.v3.patch > > > Based on the mailing list discussion at > https://lists.apache.org/thread.html/81c633cbe1f6f78421cbdad5b9549643c67803a723a9d86a513264c0@%3Cdev.hbase.apache.org%3E > it sounds like we should record some of the thoughts for future contributors > to refer to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257437#comment-16257437
]
Sean Busbey commented on HBASE-19289:
-
The more I think about this, the more I think changing LocalFileSystem needs to
be the answer. unless we want to expressly tell folks not to run HBase on top
of LocalFileSystem, in which case why are we running tests against it in the
first place?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257476#comment-16257476
]
Samarth Jain edited comment on HBASE-18359 at 11/17/17 7:50 PM:
bq. On older versions also it is not as the
CoprocessorHConnection#getConnectionForEnvironment was taking a
CoprocessorEnvironment instance not config
The issue is present in older versions. Instead of using the env passed in, the
code was using the environment of HRegionServer which is what the description
also talks about.
{code}
if (env instanceof RegionCoprocessorEnvironment) {
RegionCoprocessorEnvironment e = (RegionCoprocessorEnvironment) env;
RegionServerServices services = e.getRegionServerServices();
if (services instanceof HRegionServer) {
return new CoprocessorHConnection((HRegionServer) services);
}
}
{code}
bq. What we need in HBase is an API
CoprocessorEnvironment#getConnection(Config). This call will always make new
connection (which is short circuit enabled). So caching of this and reuse the
callee has to take care. Is that ok?
I think that should work, [~anoop.hbase]. We already cache the HConnection in
CoprocessorHConnectionTableFactory by doing this:
{code}
private synchronized HConnection getConnection(Configuration conf) throws
IOException {
if (connection == null || connection.isClosed()) {
connection = new CoprocessorHConnection(conf, server);
}
return connection;
}
{code}
It would be good if the API has explicit documentation saying it is the
caller's responsibility to make sure the connection returned by the
getConnection(config) API is appropriately closed.
was (Author: samarthjain):
bq. On older versions also it is not as the
CoprocessorHConnection#getConnectionForEnvironment was taking a
CoprocessorEnvironment instance not config
The issue is present in older versions. Instead of using the env passed in, the
code was using the environment of HRegionServer which is what the description
also talks about.
{code
if (env instanceof RegionCoprocessorEnvironment) {
RegionCoprocessorEnvironment e = (RegionCoprocessorEnvironment) env;
RegionServerServices services = e.getRegionServerServices();
if (services instanceof HRegionServer) {
return new CoprocessorHConnection((HRegionServer) services);
}
}
{code}
bq. What we need in HBase is an API
CoprocessorEnvironment#getConnection(Config). This call will always make new
connection (which is short circuit enabled). So caching of this and reuse the
callee has to take care. Is that ok?
I think that should work, [~anoop.hbase]. We already cache the HConnection in
CoprocessorHConnectionTableFactory by doing this:
{code}
private synchronized HConnection getConnection(Configuration conf) throws
IOException {
if (connection == null || connection.isClosed()) {
connection = new CoprocessorHConnection(conf, server);
}
return connection;
}
{code}
It would be good if the API has explicit documentation saying it is the
caller's responsibility to make sure the connection returned by the
getConnection(config) API is appropriately closed.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
>
[jira] [Updated] (HBASE-19269) Reenable TestShellRSGroups
[ https://issues.apache.org/jira/browse/HBASE-19269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19269: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.0.0) 2.0.0-beta-1 Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thank you for the fixup [~andrewcheng] > Reenable TestShellRSGroups > -- > > Key: HBASE-19269 > URL: https://issues.apache.org/jira/browse/HBASE-19269 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Guangxu Cheng > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19269.master.001.patch, > HBASE-19269.master.002.patch > > > It was disabled by the parent issue because RSGroups was failing. RSGroups > now works but this test is still failling. Need to dig in (signal from these > jruby tests is murky). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19123) Purge 'complete' support from Coprocesor Observers
[ https://issues.apache.org/jira/browse/HBASE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19123: -- Attachment: HBASE-19123.master.004.patch > Purge 'complete' support from Coprocesor Observers > -- > > Key: HBASE-19123 > URL: https://issues.apache.org/jira/browse/HBASE-19123 > Project: HBase > Issue Type: Task > Components: Coprocessors >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19123.master.001.patch, > HBASE-19123.master.002.patch, HBASE-19123.master.003.patch, > HBASE-19123.master.004.patch > > > Up on dev list under '[DISCUSSION] Removing the bypass semantic from the > Coprocessor APIs', we are discussing purge of 'complete'. Unless objection, > lets purge for beta-1. > [~andrew.purt...@gmail.com] says the following up on the dev list: > It would simplify the theory of operation for coprocessors if we can assume > either the entire chain will complete or one of the coprocessors in the chain > will throw an exception that not only terminates processing of the rest of > the chain but also the operation in progress. > Security coprocessors interrupt processing by throwing an exception, which is > meant to propagate all the way back to the user. > I think it's more than fair to ask the same question about 'complete' as we > did about 'bypass': Does anyone use it? Is it needed? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19204) branch-1.2 times out and is taking 6-7 hours to complete
[ https://issues.apache.org/jira/browse/HBASE-19204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257417#comment-16257417 ] stack commented on HBASE-19204: --- Can't commit because branch-1 is unstable so can't tell if this destabilizes or not. > branch-1.2 times out and is taking 6-7 hours to complete > > > Key: HBASE-19204 > URL: https://issues.apache.org/jira/browse/HBASE-19204 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack > > Sean has been looking at tooling and infra. This Umbrellas is about looking > at actual tests. For example, running locally on dedicated machine I picked a > random test, TestPerColumnFamilyFlush. In my test run, it wrote 16M lines. It > seems to be having zk issues but it is catching interrupts and ignoring them > ([~carp84] fixed this in later versions over in HBASE-18441). > Let me try and do some fixup under this umbrella so we can get a 1.2.7 out > the door. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16574) Add backup / restore feature to refguide
[ https://issues.apache.org/jira/browse/HBASE-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257458#comment-16257458 ] Vladimir Rodionov commented on HBASE-16574: --- [~elserj], can you attach pdf? > Add backup / restore feature to refguide > > > Key: HBASE-16574 > URL: https://issues.apache.org/jira/browse/HBASE-16574 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Frank Welsch > Labels: backup > Fix For: 2.0.0-beta-1 > > Attachments: B command-line tools and configuration (updated).pdf, > Backup-and-Restore-Apache_19Sep2016.pdf, HBASE-16574.001.patch, > HBASE-16574.002.patch, HBASE-16574.003.branch-2.patch, > HBASE-16574.004.branch-2.patch, HBASE-16574.005.branch-2.patch, > HBASE-16574.006.branch-2.patch, HBASE-16574.007.branch-2.patch, > HBASE-16574.008.branch-2.patch, HBASE-16574.009.branch-2.patch, > apache_hbase_reference_guide_004.pdf, apache_hbase_reference_guide_007.pdf, > apache_hbase_reference_guide_008.pdf, hbase-book-16574.003.pdf, > hbase_reference_guide.v1.pdf > > > This issue is to add backup / restore feature description to hbase refguide. > The description should cover: > scenarios where backup / restore is used > backup / restore commands and sample usage > considerations in setup -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16574) Add backup / restore feature to refguide
[
https://issues.apache.org/jira/browse/HBASE-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257467#comment-16257467
]
Hadoop QA commented on HBASE-16574:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}150m
23s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-16574 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898234/HBASE-16574.009.branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 404029bdc2e5 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 63b02a072a |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9894/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9894/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Add backup / restore feature to refguide
>
>
> Key: HBASE-16574
> URL: https://issues.apache.org/jira/browse/HBASE-16574
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Frank Welsch
> Labels: backup
> Fix For: 2.0.0-beta-1
>
> Attachments: B command-line tools and configuration (updated).pdf,
> Backup-and-Restore-Apache_19Sep2016.pdf, HBASE-16574.001.patch,
> HBASE-16574.002.patch, HBASE-16574.003.branch-2.patch,
> HBASE-16574.004.branch-2.patch, HBASE-16574.005.branch-2.patch,
> HBASE-16574.006.branch-2.patch, HBASE-16574.007.branch-2.patch,
> HBASE-16574.008.branch-2.patch, HBASE-16574.009.branch-2.patch,
> apache_hbase_reference_guide_004.pdf, apache_hbase_reference_guide_007.pdf,
> apache_hbase_reference_guide_008.pdf, hbase-book-16574.003.pdf,
> hbase_reference_guide.v1.pdf
>
>
> This issue is to add backup / restore feature description to hbase refguide.
> The description should cover:
> scenarios where backup / restore is used
> backup / restore commands and sample usage
> considerations in setup
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Wingert updated HBASE-15320: - Attachment: HBASE-15320.pdf > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.2.patch, > HBASE-15320.master.3.patch, HBASE-15320.master.4.patch, > HBASE-15320.master.5.patch, HBASE-15320.master.6.patch, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18440) ITs and Actions modify immutable TableDescriptors
[ https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18440: -- Fix Version/s: (was: 2.0.0) 2.0.0-beta-1 > ITs and Actions modify immutable TableDescriptors > - > > Key: HBASE-18440 > URL: https://issues.apache.org/jira/browse/HBASE-18440 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18440.patch, HBASE-18440.v2.patch, > HBASE-18440.v3.patch, HBASE-18440.v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18988) Add release managers to reference guide
[ https://issues.apache.org/jira/browse/HBASE-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257262#comment-16257262 ] stack commented on HBASE-18988: --- [~syuanjiang] has disappeared > Add release managers to reference guide > --- > > Key: HBASE-18988 > URL: https://issues.apache.org/jira/browse/HBASE-18988 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Peter Somogyi >Priority: Trivial > Labels: beginner > > Reference guide lists release managers only up to version 1.3. We should have > a complete list there. > http://hbase.apache.org/book.html#_release_managers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257283#comment-16257283 ] Anoop Sam John commented on HBASE-19092: All the CPs get the type as Cell not ExtendedCell. Now if we add APIs in ExtendedCell alone, CP users has to know that even if type is Cell all objects are of type ExtendedCell and then call with type casting. This will look as a bad design IMHO. Till now they could get tags using CellUtil API. This will be far better (I mean using any static method).. I agree add to ExtendedCell/RawCell is the best way. But then we have to make sure it is ExtendedCell/ RawCell type flowing through this server side CPs. We cannot do that at least as of now. That is why thinking ExtendedCell is not the right place. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19123) Purge 'complete' support from Coprocesor Observers
[ https://issues.apache.org/jira/browse/HBASE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257354#comment-16257354 ] stack commented on HBASE-19123: --- .004 Retry. The tests pass locally. > Purge 'complete' support from Coprocesor Observers > -- > > Key: HBASE-19123 > URL: https://issues.apache.org/jira/browse/HBASE-19123 > Project: HBase > Issue Type: Task > Components: Coprocessors >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19123.master.001.patch, > HBASE-19123.master.002.patch, HBASE-19123.master.003.patch, > HBASE-19123.master.004.patch > > > Up on dev list under '[DISCUSSION] Removing the bypass semantic from the > Coprocessor APIs', we are discussing purge of 'complete'. Unless objection, > lets purge for beta-1. > [~andrew.purt...@gmail.com] says the following up on the dev list: > It would simplify the theory of operation for coprocessors if we can assume > either the entire chain will complete or one of the coprocessors in the chain > will throw an exception that not only terminates processing of the rest of > the chain but also the operation in progress. > Security coprocessors interrupt processing by throwing an exception, which is > meant to propagate all the way back to the user. > I think it's more than fair to ask the same question about 'complete' as we > did about 'bypass': Does anyone use it? Is it needed? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257376#comment-16257376 ] Appy commented on HBASE-19271: -- +1 Reworded a bit on committing. > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19271: - Fix Version/s: (was: 3.0.0) 2.0.0-beta-1 > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19290) Reduce zk request when doing split log
[ https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19290: - Status: Patch Available (was: Open) > Reduce zk request when doing split log > -- > > Key: HBASE-19290 > URL: https://issues.apache.org/jira/browse/HBASE-19290 > Project: HBase > Issue Type: Improvement >Reporter: binlijin >Assignee: binlijin > Attachments: HBASE-19290.master.001.patch > > > We observe once the cluster has 1000+ nodes and when hundreds of nodes abort > and doing split log, the split is very very slow, and we find the > regionserver and master wait on the zookeeper response, so we need to reduce > zookeeper request and pressure for big cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257397#comment-16257397
]
Sean Busbey commented on HBASE-19289:
-
How would you propose relaxing it?
Two solutions I see:
* updating LocalFileSystem to support hflush/hsync
* telling CommonFSUtils to ignore the check on LocalFileSystem
The former is probably going to be easier due to the way we wrap FileSystems.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257404#comment-16257404
]
Ted Yu commented on HBASE-19289:
bq. updating LocalFileSystem to support hflush/hsync
This would be done in hadoop, right ?
Consider logging a HADOOP issue.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257404#comment-16257404
]
Ted Yu edited comment on HBASE-19289 at 11/17/17 7:07 PM:
--
bq. updating LocalFileSystem to support hflush/hsync
This would be done in hadoop, right ?
Consider logging a HADOOP issue - possibly after finding out which commit(s)
changed the behavior in hadoop3.
was (Author: yuzhih...@gmail.com):
bq. updating LocalFileSystem to support hflush/hsync
This would be done in hadoop, right ?
Consider logging a HADOOP issue.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18974) Document "Becoming a Committer"
[ https://issues.apache.org/jira/browse/HBASE-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257419#comment-16257419 ] Misty Stanley-Jones commented on HBASE-18974: - I built with my updates and it looks good to me locally. Be aware that the links to top-level site pages won't work on the local build, but this is known. > Document "Becoming a Committer" > --- > > Key: HBASE-18974 > URL: https://issues.apache.org/jira/browse/HBASE-18974 > Project: HBase > Issue Type: Bug > Components: community, documentation >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18974-copyedit-addendum.patch, HBASE-18974.patch, > HBASE-18974.v2.patch, HBASE-18974.v3.patch > > > Based on the mailing list discussion at > https://lists.apache.org/thread.html/81c633cbe1f6f78421cbdad5b9549643c67803a723a9d86a513264c0@%3Cdev.hbase.apache.org%3E > it sounds like we should record some of the thoughts for future contributors > to refer to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
[ https://issues.apache.org/jira/browse/HBASE-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257448#comment-16257448 ] Appy commented on HBASE-19114: -- Pretty sure all the failures are flukes - different tests failing, and passing when run locally. :-/ Let's wait for next precommit. > Split out o.a.h.h.zookeeper from hbase-server and hbase-client > -- > > Key: HBASE-19114 > URL: https://issues.apache.org/jira/browse/HBASE-19114 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19114.master.001.patch, > HBASE-19114.master.002.patch, HBASE-19114.master.003.patch, > HBASE-19114.master.004.patch, HBASE-19114.master.005.patch, > HBASE-19114.master.006.patch, HBASE-19114.master.007.patch, > HBASE-19114.master.008.patch > > > Changes so far: > - Moved DrainingServerTracker and RegionServerTracker to > hbase-server:o.a.h.h.master. > - Moved SplitOrMergeTracker to oahh.master (because it depends on a PB) > - Moving hbase-client:oahh.zookeeper.* to hbase-zookeeper module. After > [~Apache9]'s cleanup (HBASE-19200), hbase-client doesn't need them anymore > (except 3 classes). > - Renamed some classes to use a consistent naming for classes - ZK instead of > mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: > MiniZooKeeperCluster, ZooKeeperConnectionException. Left RecoverableZooKeeper > for lack of better name. (suggestions?) > - Sadly, can't move tests out because they depend on HBaseTestingUtility > (which defeats part of the purpose - trimming down hbase-server tests. We > need to promote more use of mocks in our tests) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257494#comment-16257494 ] Mike Wingert commented on HBASE-15320: -- I've attached some docs, let me know if they are not detailed enough. > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.2.patch, > HBASE-15320.master.3.patch, HBASE-15320.master.4.patch, > HBASE-15320.master.5.patch, HBASE-15320.master.6.patch, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257416#comment-16257416 ] stack commented on HBASE-18090: --- Yeah, not you. I forgot for a sec. Our branch-1 is sick. HBASE-19204. Let me add this to a list of issues to commit once branch-1 stabilizes again. > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Mikhail Antonov >Assignee: xinxin fan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18090-V3-master.patch, > HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, > HBASE-18090-branch-1-v2.patch, HBASE-18090-branch-1-v2.patch, > HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch, > HBASE-18090.branch-1.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257476#comment-16257476
]
Samarth Jain commented on HBASE-18359:
--
bq. On older versions also it is not as the
CoprocessorHConnection#getConnectionForEnvironment was taking a
CoprocessorEnvironment instance not config
The issue is present in older versions. Instead of using the env passed in, the
code was using the environment of HRegionServer which is what the description
also talks about.
{code
if (env instanceof RegionCoprocessorEnvironment) {
RegionCoprocessorEnvironment e = (RegionCoprocessorEnvironment) env;
RegionServerServices services = e.getRegionServerServices();
if (services instanceof HRegionServer) {
return new CoprocessorHConnection((HRegionServer) services);
}
}
{code}
bq. What we need in HBase is an API
CoprocessorEnvironment#getConnection(Config). This call will always make new
connection (which is short circuit enabled). So caching of this and reuse the
callee has to take care. Is that ok?
I think that should work, [~anoop.hbase]. We already cache the HConnection in
CoprocessorHConnectionTableFactory by doing this:
{code}
private synchronized HConnection getConnection(Configuration conf) throws
IOException {
if (connection == null || connection.isClosed()) {
connection = new CoprocessorHConnection(conf, server);
}
return connection;
}
{code}
It would be good if the API has explicit documentation saying it is the
caller's responsibility to make sure the connection returned by the
getConnection(config) API is appropriately closed.
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 2.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18938) Backport HBASE-16985 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18938: -- Attachment: HBASE-18938-branch-1.3.v2.patch Fixed checkstyle warning. > Backport HBASE-16985 to branch-1.3 > -- > > Key: HBASE-18938 > URL: https://issues.apache.org/jira/browse/HBASE-18938 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18938-branch-1.3.patch, > HBASE-18938-branch-1.3.patch, HBASE-18938-branch-1.3.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18988) Add release managers to reference guide
[ https://issues.apache.org/jira/browse/HBASE-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257280#comment-16257280 ] Peter Somogyi commented on HBASE-18988: --- Thank you for the clarification. > Add release managers to reference guide > --- > > Key: HBASE-18988 > URL: https://issues.apache.org/jira/browse/HBASE-18988 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Peter Somogyi >Priority: Trivial > Labels: beginner > > Reference guide lists release managers only up to version 1.3. We should have > a complete list there. > http://hbase.apache.org/book.html#_release_managers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257323#comment-16257323 ] stack commented on HBASE-19092: --- bq. But we can not chnage the server to work with ExtCell now. Right. As per [~chia7712], it is too late to make such changes (so I am inclined to just have users cast to ExtendedCell at this stage in the game if they want to play with Tags in a CP). getTags is in CellUtil but it is deprecated in 2.0.0 to be removed in 3.0.0. It does not point to an alternative. Where should the alternative be? Tag#get(Cell) might be best for now at this late stage but I should be able to do Cell#getTags or at least ExtendedCell#getTags? That would be most natural. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19285) Add per-table latency histograms
[ https://issues.apache.org/jira/browse/HBASE-19285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257189#comment-16257189 ] Josh Elser commented on HBASE-19285: bq. Wouldn't these be as expensive as the per region latency histogram metrics were? Since we have to track tables region by region and roll up? Thanks for the questions! What I currently have staged (working through connecting all the dots through the hadoop* indirection :P) would be tracking these by table instead of rolling up Regions. My plan is to re-run the perf testing that Enis mentioned over on HBASE-17017. My _hope_ is that by having fewer histograms, we'll see some amortization of the cost but, admittedly, I'm not optimistic. My expectation is that having something in front of me that can show the bottleneck, a better solution will eventually present itself. In some of those older issues, Enis did mention how doing the aggregations in a separate through would be better, but I'm not sure how we would do that for op latencies. Taking the advice to try out HdrHistogram (or some other library) is also rolling around in my head. I assume that this will be a tricky one to work out :) At the end of the day, and echoing some comments offline by Clay, operators lost some really nice information by the removal of the per-region histos. For multi-tenant installations, there's no longer a mechanism to track "typical" performance characteristics for a table which makes it impossible to know when performance is "atypical" (and something that should be alerted on). > Add per-table latency histograms > > > Key: HBASE-19285 > URL: https://issues.apache.org/jira/browse/HBASE-19285 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Clay B. >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.3 > > > HBASE-17017 removed the per-region latency histograms (e.g. Get, Put, Scan at > p75, p85, etc) > HBASE-15518 added some per-table metrics, but not the latency histograms. > Given the previous conversations, it seems like it these per-table > aggregations weren't intentionally omitted, just never re-implemented after > the per-region removal. They're some really nice out-of-the-box metrics we > can provide to our users/admins as long as it's not detrimental. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19291) Use common header and footer for JSP pages
[ https://issues.apache.org/jira/browse/HBASE-19291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257253#comment-16257253 ] Appy commented on HBASE-19291: -- Ping [~Apache9] / [~elserj] (since you appear in recent history of web-apps dir :) ) > Use common header and footer for JSP pages > -- > > Key: HBASE-19291 > URL: https://issues.apache.org/jira/browse/HBASE-19291 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19291.master.001.patch > > > Use header and footer in our *.jsp pages to avoid unnecessary redundancy > (copy-paste of code) > (Been sitting in my local repo for long, best to get following pesky > user-facing things fixed before the next major release) > Misc edits: > - Due to redundancy, new additions make it to some places but not others. For > eg there are missing links to "/logLevel", "/processRS.jsp" in few places. > - Fix processMaster.jsp wrongly pointing to rs-status instead of > master-status (probably due to copy paste from processRS.jsp) > - Deleted a bunch of extraneous "" in processMaster.jsp & processRS.jsp > - Added missing tag in snapshot.jsp > - Deleted fossils of html5shiv.js. It's uses and the js itself were deleted > in the commit "819aed4ccd073d818bfef5931ec8d248bfae5f1f" > - Fixed wrongly matched heading tags > - Deleted some unused variables > Tested: > Ran standalone cluster and opened each page to make sure it looked right. > Sidenote: > Looks like HBASE-3835 started the work of converting from jsp to jamon, but > the work didn't finish. Now we have a mix of jsp and jamon. Needs > reconciling, but later. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18938) Backport HBASE-16985 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-18938: - Assignee: Peter Somogyi (was: Ashish Singhi) > Backport HBASE-16985 to branch-1.3 > -- > > Key: HBASE-18938 > URL: https://issues.apache.org/jira/browse/HBASE-18938 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Peter Somogyi > Fix For: 1.3.2 > > Attachments: HBASE-18938-branch-1.3.patch, > HBASE-18938-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257290#comment-16257290 ] stack commented on HBASE-19092: --- bq. All the CPs get the type as Cell not ExtendedCell. What is the suggestion then? Adding getTags(Cell) to Tags? CPs will still get an ExtendedCellBuilder instance right for case where they want to create their own Cells? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257298#comment-16257298 ] Anoop Sam John edited comment on HBASE-19092 at 11/17/17 5:51 PM: -- Ya for creation of Cells, they will get an ExtCellBuilder type (One which is having way to add Tags). getTags() also the best place is ExtendedCell/ RawCell.. But we can not chnage the server to work with ExtCell now. :-( Ya we can have Tags and add the static APIs like getTags(Cell), getTag(Cell, type) there? BTW we having these getTags() API in CellUtil, so we cannot really remove them now. We will deprecate for 2.0 right? was (Author: anoop.hbase): Ya for creation of Cells, they will get an ExtCellBuilder type (One which is having way to add Tags). getTags() also the best place is ExtendedCell/ RawCell.. But we can not chnage the server to work with ExtCell now. :-( Ya we can have Tags and add the static APIs like getTags(Cell), getTag(Cell, type) there? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI
[ https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257302#comment-16257302 ] stack commented on HBASE-17436: --- Thank you [~gubjanos] Can you say what is in the json that is not already present in the UI? Thanks (I appreciate your effort at exposing more info to the operator) > Add facility to provide more information for Other Regions seen on Master UI > > > Key: HBASE-17436 > URL: https://issues.apache.org/jira/browse/HBASE-17436 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Janos Gub > Labels: ui, usability > Fix For: 2.0.0 > > Attachments: 17436.v10.txt, 17436.v9.txt, HBASE-17436-v2.patch, > HBASE-17436-v5.patch, HBASE-17436-v6.patch, HBASE-17436-v6.patch, > HBASE-17436-v7.patch, HBASE-17436-v8.patch, HBASE-17436.patch, > HBASE-17779-v3.patch, HBASE-17779-v4.patch, Screen Shot 2017-04-24 at 8.43.31 > PM.png, Screen Shot 2017-04-26 at 4.39.49 PM.png, Screen Shot 2017-11-08 at > 10.29.49 AM.png, Screen Shot 2017-11-08 at 10.30.09 AM.png, initial.patch, > regionstates.png, table-selected.png, tableregions.png > > > [~rpednekar] and I were looking at a case where the count displayed under > Other Regions was high (~1200). > Since the table page just maintains a count instead of List of region names, > it is very difficult for user to determine which regions belong to this > category. > We should provide facility to provide more details for this category (LOG, > JMX, etc). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19285) Add per-table latency histograms
[ https://issues.apache.org/jira/browse/HBASE-19285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257339#comment-16257339 ] Josh Elser commented on HBASE-19285: Awesome. Should have some results here shortly :) Thanks again for the input. > Add per-table latency histograms > > > Key: HBASE-19285 > URL: https://issues.apache.org/jira/browse/HBASE-19285 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Clay B. >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.3 > > > HBASE-17017 removed the per-region latency histograms (e.g. Get, Put, Scan at > p75, p85, etc) > HBASE-15518 added some per-table metrics, but not the latency histograms. > Given the previous conversations, it seems like it these per-table > aggregations weren't intentionally omitted, just never re-implemented after > the per-region removal. They're some really nice out-of-the-box metrics we > can provide to our users/admins as long as it's not detrimental. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18938) Backport HBASE-16985 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257223#comment-16257223 ] Ashish Singhi commented on HBASE-18938: --- [~psomogyi], Do you want to take this up ? If yes, please feel free to assign yourself. I am checking another issue at the moment. Thanks. > Backport HBASE-16985 to branch-1.3 > -- > > Key: HBASE-18938 > URL: https://issues.apache.org/jira/browse/HBASE-18938 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 1.3.2 > > Attachments: HBASE-18938-branch-1.3.patch, > HBASE-18938-branch-1.3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257298#comment-16257298 ] Anoop Sam John commented on HBASE-19092: Ya for creation of Cells, they will get an ExtCellBuilder type (One which is having way to add Tags). getTags() also the best place is ExtendedCell/ RawCell.. But we can not chnage the server to work with ExtCell now. :-( Ya we can have Tags and add the static APIs like getTags(Cell), getTag(Cell, type) there? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092-branch-2_5.patch, HBASE-19092-branch-2_5.patch, > HBASE-19092.branch-2.0.02.patch, HBASE-19092_001-branch-2.patch, > HBASE-19092_001.patch, HBASE-19092_002-branch-2.patch, HBASE-19092_002.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19285) Add per-table latency histograms
[ https://issues.apache.org/jira/browse/HBASE-19285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257322#comment-16257322 ] Andrew Purtell commented on HBASE-19285: I'm all for getting back table granularity histograms if they are not going to be too expensive. > Add per-table latency histograms > > > Key: HBASE-19285 > URL: https://issues.apache.org/jira/browse/HBASE-19285 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Clay B. >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.3 > > > HBASE-17017 removed the per-region latency histograms (e.g. Get, Put, Scan at > p75, p85, etc) > HBASE-15518 added some per-table metrics, but not the latency histograms. > Given the previous conversations, it seems like it these per-table > aggregations weren't intentionally omitted, just never re-implemented after > the per-region removal. They're some really nice out-of-the-box metrics we > can provide to our users/admins as long as it's not detrimental. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
[ https://issues.apache.org/jira/browse/HBASE-19271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257333#comment-16257333 ] stack commented on HBASE-19271: --- +1 > Update ref guide about the async client to reflect the change in HBASE-19251 > > > Key: HBASE-19271 > URL: https://issues.apache.org/jira/browse/HBASE-19271 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-19271.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16574) Add backup / restore feature to refguide
[ https://issues.apache.org/jira/browse/HBASE-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-16574: --- Attachment: HBASE-16574.009.branch-2.patch .009 Adds the hfilecleaner, removes trailing whitespace on lines, and s/key/name/ in the Hadoop Configuration xml blocks. I think this is good now? > Add backup / restore feature to refguide > > > Key: HBASE-16574 > URL: https://issues.apache.org/jira/browse/HBASE-16574 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Frank Welsch > Labels: backup > Fix For: 2.0.0-beta-1 > > Attachments: B command-line tools and configuration (updated).pdf, > Backup-and-Restore-Apache_19Sep2016.pdf, HBASE-16574.001.patch, > HBASE-16574.002.patch, HBASE-16574.003.branch-2.patch, > HBASE-16574.004.branch-2.patch, HBASE-16574.005.branch-2.patch, > HBASE-16574.006.branch-2.patch, HBASE-16574.007.branch-2.patch, > HBASE-16574.008.branch-2.patch, HBASE-16574.009.branch-2.patch, > apache_hbase_reference_guide_004.pdf, apache_hbase_reference_guide_007.pdf, > apache_hbase_reference_guide_008.pdf, hbase-book-16574.003.pdf, > hbase_reference_guide.v1.pdf > > > This issue is to add backup / restore feature description to hbase refguide. > The description should cover: > scenarios where backup / restore is used > backup / restore commands and sample usage > considerations in setup -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19293) Support add a disabled state replication peer directly
[ https://issues.apache.org/jira/browse/HBASE-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257219#comment-16257219 ] Ashish Singhi commented on HBASE-19293: --- Two comments, 1. It will be good to capture this replicate state in the audit log. 2. Need to update the document section, http://hbase.apache.org/book.html#hbase.replication.management > Support add a disabled state replication peer directly > -- > > Key: HBASE-19293 > URL: https://issues.apache.org/jira/browse/HBASE-19293 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19293.master.001.patch > > > Now when add a replication peer, the default state is enabled. If you want > add a disabled replication peer, you need add a peer first, then disable it. > It need two step to finish now. > Use case for add a disabled replication peer. When user want sync data from a > cluster A to a new peer cluster. > 1. Add a disabled replication peer. And config the table to peer config. > 2. Take a snapshot of table and export snapshot to peer cluster. > 3. Restore snapshot in peer cluster. > 4. Enable the peer and wait all stuck replication log replicated to peer > cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
