[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114894#comment-14114894 ] stack commented on HBASE-11165: --- bq. I thought that the primary requirement for splittable meta is not really read-write throughput, but rather the thinking that on large enough cluster with 1M+ regions, meta may not simply fit in master memory (or JVM would have hard time keeping process consuming that much memory up)? It'd be both [~mantonov] Scale reads because pieces of meta served by many servers and master can write in // so scale writes too (though not all ops will be //izable). Yeah, adding features to meta comes up all the time (today it was adding region flush/compaction history, previous its keeping hfiles per cf in meta, region replica info, and so on) bq. I would think that if we want to keep regions as a unit of assignment/recovery, splittable meta is a the must (so far didn't see an approach describing how to avoid it) bq. HBASE-10070 is about region replicas I was thinking that we could have meta region replicas to scale out the read i/o using hbase-10070 (solving one of the objections to splitting meta arguments) bq. HBASE-11467 is about ZK-less client... That is the issue where client takes a quorum of masters instead of a quorum of zks, right? I was extrapolating that the endpoint of this issue is a quorum of masters where we could read from any (or at least some reads could be stale...) as another means of scaling out the read i/o when master and meta colocated. bq. ...but it would work fine with current single active-many backup-masters schema as well. Good to know [~mantonov] bq. I'm thinking that multi-masters and partitioned-masters (if we go in these approaches) need to be discussed closely together and considering each other, otherwise it'd be really hard to merge them together later on. Agree. This issue seems to be arriving at single master to serve mllions of regions. A quorum of masters or partitioning master responsibilities for sure should be discussed together but I don't think they are soln to this issues problem (maybe partitioned master but single server soln seems simpler?) bq. I'd be curious to hear more opinions/assessments on how bad is that when master is down, and what timeframe various people would consider as generally ok, kind of long, really want it to be faster and unacceptably long? Yes. Will write something up. In fact I don't even think a client can connect to the cluster currently if master is down which makes a bit of a farce of the above notion and needs fixing. Let me look at HBASE-7767. I got burned by it today. Its annoying to say the least. Yeah, that seems to be the conclusion that is beginning to prevail here. Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HBASE-11726:
--
Attachment: HBASE-11726.v1.diff
Yeah, good idea [~srikanth235] new patch points to the HBase book URL.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114898#comment-14114898
]
Srikanth Srungarapu commented on HBASE-11726:
-
+1 (non-binding)
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HBASE-11726:
--
Affects Version/s: 2.0.0
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11860) Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
Esteban Gutierrez created HBASE-11860: - Summary: Restore UpgradeTo in order to upgrade a HBase cluster to the latest release Key: HBASE-11860 URL: https://issues.apache.org/jira/browse/HBASE-11860 Project: HBase Issue Type: Improvement Components: HFile, master Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Fix For: 0.99.0, 2.0.0 Users should have the option to upgrade to a newer release of HBase without performing a transitive upgrade to HBase 0.96 (EOL'd) or HBase 0.98 and then HBase 1.x or 2.x In HBASE-11611 UpgradeTo96 was removed and it won't be possible to migrate a HBase 0.94 cluster to HBase 1.x or 2.x without the extra step. Initially we can restore part of the functionality of the tool supporting migration to namespaces and report for the existence of any HFileV1 file and ZKDataMigrator could be safely removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114917#comment-14114917
]
Esteban Gutierrez commented on HBASE-11726:
---
Created HBASE-11860 to restore the the upgrade tool for releases post HBase
1.x. so the user can perform the upgrade step.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114920#comment-14114920 ] Mikhail Antonov commented on HBASE-11165: - bq. That is the issue where client takes a quorum of masters instead of a quorum of zks, right? I was extrapolating that the endpoint of this issue is a quorum of masters where we could read from any (or at least some reads could be stale...) as another means of scaling out the read i/o when master and meta colocated. Yes, that's this one. Yeah, it should help with scaling read I/O (which, I believe, is a vast majority of meta access calls? by the way, could somebody estimate the distribution or read vs writes to meta table, in terms of requests per second/networking traffic/disk access? are there metrics for that?) bq. This issue seems to be arriving at single master to serve mllions of regions I believe there are 2 dimensions here, right? - For multi-masters (replicated master) the objection I believe is to 1) maintain up-to-date in-memory state on 1 master, thus avoiding startup cost for second master (that's actually why I solicited the estimates of how long does the full restart takes now on that big cluster) and 2) scale out reads by serving them out of copies located at different machines. But multi-masters does not solve problem when meta doesn't fit in memory, of when writes need to scale better - partitioned masters, in turn, address this second aspect, fitting meta in memory and scaling writes. Does it look like accurate summary to you? The viable combined solution might be multi-master setup, when masters are serving split meta and grouped by meta region replicas (like, HM1 and HM2 are serving replicas of metaRegion1, metaRegion2, MetaRegion3, and HM3 and HM4 are serving replicas of metaRegion4 and metaRegion5? with masters being a region server now, having really many masters in the cluster might be just right direction to go with?) bq. In fact I don't even think a client can connect to the cluster currently if master is down which makes a bit of a farce of the above notion and needs fixing. That probably is an argument for multi-master layout too :) bq. Let me look at HBASE-7767 That would be great. I also did a first pass to review the patch on the review board and planning to get back to get closer look. Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11811) Use binary search for seeking into a block
[ https://issues.apache.org/jira/browse/HBASE-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11811: -- Attachment: 11811-wip-v4.txt Slightly updated version. In all my local testing I see great improvement for Gets and no measurable penalty for Scans. In the worst case (if each block only receives a single Get request) we can expect that on average we now look through twice as many KV. If the block is indeed loaded from HDFS each time, this is negligible. I'll try to test with a real cluster and PE soon. Use binary search for seeking into a block -- Key: HBASE-11811 URL: https://issues.apache.org/jira/browse/HBASE-11811 Project: HBase Issue Type: Brainstorming Reporter: Lars Hofhansl Attachments: 11811-wip-v2.txt, 11811-wip-v4.txt, block_index-v2.txt Currently upon every seek (including Gets) we need to linearly look through the block from the beginning until we find the Cell we are looking for. It should be possible to build a simple cache of offsets of Cells for each block as it is loaded and then use binary search to find the Cell in question. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[
https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114946#comment-14114946
]
Hadoop QA commented on HBASE-11777:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665204/HBASE-11777_V2.patch
against trunk revision .
ATTACHMENT ID: 12665204
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityWithCheckAuths
org.apache.hadoop.hbase.client.TestReplicaWithCluster
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.client.TestScannersFromClientSide
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScanBase.testScan(TestTableInputFormatScanBase.java:238)
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.testScanEmptyToBBA(TestTableInputFormatScan1.java:70)
at
org.apache.hadoop.hdfs.TestFileCorruption.testArrayOutOfBoundsException(TestFileCorruption.java:130)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10632//console
This message is automatically generated.
Find a way to use KV.setSequenceId() on Cells on the server-side read path
--
Key: HBASE-11777
URL: https://issues.apache.org/jira/browse/HBASE-11777
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 2.0.0
Attachments: CellWithSequenceNumber.java, HBASE-11777.patch,
HBASE-11777_V2.patch
Over in HBASE-11591 there was a need to set the sequenceId of the HFile to
the bulk loaded KVs. Since we are trying to use the concept of Cells in the
read path if we need to use setSequenceId(), then the Cell has to be
converted to KV and only KeyValue impl has the accessor setSequenceId().
[~anoop.hbase] suggested if we can use a Server side impl of Cell and have
these accessors in them.
This JIRA aims to solve this and see the related code changes that needs to
be carried out for this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114951#comment-14114951
]
Hadoop QA commented on HBASE-11726:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665263/HBASE-11726.v1.diff
against trunk revision .
ATTACHMENT ID: 12665263
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10635//console
This message is automatically generated.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11811) Use binary search for seeking into a block
[
https://issues.apache.org/jira/browse/HBASE-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114952#comment-14114952
]
Hadoop QA commented on HBASE-11811:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665272/11811-wip-v4.txt
against trunk revision .
ATTACHMENT ID: 12665272
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ blockBuffer.position(blockBuffer.position() + currKeyLen +
currValueLen + currMemstoreTSLen + KEY_VALUE_LEN_SIZE);
+//byte[] v = Bytes.toBytes(Some string that is longer than just three
characters and makes this a more real scenario...);
+//byte[] v = Bytes.toBytes(Some string that is longer than just three
characters and makes this a more real scenario...);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestClassFinder
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10636//console
This message is automatically generated.
Use binary search for seeking into a block
--
Key: HBASE-11811
URL: https://issues.apache.org/jira/browse/HBASE-11811
Project: HBase
Issue Type: Brainstorming
Reporter: Lars Hofhansl
Attachments: 11811-wip-v2.txt, 11811-wip-v4.txt, block_index-v2.txt
Currently upon every seek (including Gets) we need to linearly look through
the block from the beginning until we find the Cell we are looking for.
It should be possible to build a simple cache of offsets of Cells for each
block as it is loaded and then use binary search to find the Cell in question.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[ https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114957#comment-14114957 ] Anoop Sam John commented on HBASE-11857: Codec is also marked InterfaceAudience.Private Better change there also. Same with ReaderBase nit : Pls remove whitespaces Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) Key: HBASE-11857 URL: https://issues.apache.org/jira/browse/HBASE-11857 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Blocker Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: 11857-trunk-v1.txt ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) were removed when adding Cell codec to WAL header. Some downstream projects, such as Phoenix, depend on these methods. This JIRA is to restore these methods. Thanks to Jeffrey who found this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11643) Read and write MOB in HBase
[ https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11643: - Attachment: HBASE-11643-V16.diff Upload the patch HBASE-11643-V16.diff after changes according to Jon's comments. Thanks. Read and write MOB in HBase --- Key: HBASE-11643 URL: https://issues.apache.org/jira/browse/HBASE-11643 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, the Cells are saved in the MemStore as well, but they're flushed to elsewhere out of HBase in the format of HFiles. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[ https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114966#comment-14114966 ] Anoop Sam John commented on HBASE-11643: Yes not this patch issue Jingcheng. I have fixed this issue and pushed already so you will not see this in next QA runs. Read and write MOB in HBase --- Key: HBASE-11643 URL: https://issues.apache.org/jira/browse/HBASE-11643 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, the Cells are saved in the MemStore as well, but they're flushed to elsewhere out of HBase in the format of HFiles. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[ https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114970#comment-14114970 ] Jingcheng Du commented on HBASE-11643: -- Good to know! Thanks a lot Anoop! Read and write MOB in HBase --- Key: HBASE-11643 URL: https://issues.apache.org/jira/browse/HBASE-11643 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, the Cells are saved in the MemStore as well, but they're flushed to elsewhere out of HBase in the format of HFiles. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114976#comment-14114976
]
Hadoop QA commented on HBASE-11726:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665262/HBASE-11726.v0.diff
against trunk revision .
ATTACHMENT ID: 12665262
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10634//console
This message is automatically generated.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11837) Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no local data
[
https://issues.apache.org/jira/browse/HBASE-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114991#comment-14114991
]
Anoop Sam John commented on HBASE-11837:
+1
Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no
local data
---
Key: HBASE-11837
URL: https://issues.apache.org/jira/browse/HBASE-11837
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.98.4
Reporter: James Taylor
Assignee: Jeffrey Zhong
Attachments: HBASE-11837.patch, PHOENIX-1208-torepro.patch
Might be fixed as a byproduct of the fix for HBASE-11766.
Here's what I'm seeing:
- From an endpoint coprocessor on SYSTEM.CATALOG table
- Rows exist in the table, but are all in a single region
- Scan is a new, empty scan created like this: new Scan()
- The following scanner *does* return data:
{code}
RegionScanner scanner = region.getScanner(scan);
{code}
- The following scanner *does not* return any data:
{code}
HTableInterface hTable = env.getTable(region.getTableDesc().getTableName());
{code}
- The following scanner *does* return data:
{code}
HTablePool pool = new HTablePool (env.getConfiguration(),1);
HTableInterface hTable = pool.getTable(SYSTEM.CATALOG);
{code}
[~jeffreyz], [~apurtell]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jiajia updated HBASE-11339: -- Attachment: (was: MOB user guide.docx) HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: Umbrella Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB Design.pdf, MOB user guide.docx, hbase-11339-in-dev.patch It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jiajia updated HBASE-11339: -- Attachment: MOB user guide.docx update the mob user guide. HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: Umbrella Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB Design.pdf, MOB user guide.docx, hbase-11339-in-dev.patch It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11827) Encryption support for bulkloading data into table with encryption configured for hfile format 3
[ https://issues.apache.org/jira/browse/HBASE-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114996#comment-14114996 ] Kashif J S commented on HBASE-11827: Agree with [~apurtell] that indeed it is a security glitch. Encryption support for bulkloading data into table with encryption configured for hfile format 3 Key: HBASE-11827 URL: https://issues.apache.org/jira/browse/HBASE-11827 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Kashif J S Assignee: Kashif J S Fix For: 2.0.0, 0.98.7 Attachments: HBASE-11827-98-v1.patch, HBASE-11827-trunk-v1.patch The solution would be to add support to auto detect encryption parameters similar to other parameters like compression, datablockencoding, etc when encryption is enabled for hfile format 3. The current patch does the following: 1. Automatically detects encryption type and key in HFileOutputFormat HFileOutputFormat2. 2. Uses Base64encoder/decoder for url passing of Encryption key which is in bytes format -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11827) Encryption support for bulkloading data into table with encryption configured for hfile format 3
[ https://issues.apache.org/jira/browse/HBASE-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114997#comment-14114997 ] Kashif J S commented on HBASE-11827: Is there no alternate way apart from triggering major compaction ? Because with several TB of data, rewriting data all again doesn't seems to be a good idea :) Encryption support for bulkloading data into table with encryption configured for hfile format 3 Key: HBASE-11827 URL: https://issues.apache.org/jira/browse/HBASE-11827 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Kashif J S Assignee: Kashif J S Fix For: 2.0.0, 0.98.7 Attachments: HBASE-11827-98-v1.patch, HBASE-11827-trunk-v1.patch The solution would be to add support to auto detect encryption parameters similar to other parameters like compression, datablockencoding, etc when encryption is enabled for hfile format 3. The current patch does the following: 1. Automatically detects encryption type and key in HFileOutputFormat HFileOutputFormat2. 2. Uses Base64encoder/decoder for url passing of Encryption key which is in bytes format -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115027#comment-14115027 ] Mikhail Antonov commented on HBASE-11689: - Patch is stale, could you rebase it? Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11683) Metrics for MOB
[ https://issues.apache.org/jira/browse/HBASE-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11683: - Attachment: HBASE-11683-V3.diff Update the patch to support the metrics for the compactor to record how many cells (and size) are moved from/to mob files. Metrics for MOB --- Key: HBASE-11683 URL: https://issues.apache.org/jira/browse/HBASE-11683 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jingcheng Du Attachments: HBASE-11683-V2.diff, HBASE-11683-V3.diff, HBASE-11683.diff We need to make sure to capture metrics about mobs. Some basic ones include: # of mob writes # of mob reads # avg size of mob (?) # mob files # of mob compactions / sweeps -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11643) Read and write MOB in HBase
[ https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11643: - Attachment: HBASE-11643-V17.diff Update the patch HBASE-11643-V17.diff to move the code of deleting mob files to the DeleteTableHandler instead of using master observer. Read and write MOB in HBase --- Key: HBASE-11643 URL: https://issues.apache.org/jira/browse/HBASE-11643 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, the Cells are saved in the MemStore as well, but they're flushed to elsewhere out of HBase in the format of HFiles. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11646) Handle the MOB in compaction
[ https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11646: - Attachment: HBASE-11646-V4.diff Update the patch to refine the class import in the Compactor. Handle the MOB in compaction Key: HBASE-11646 URL: https://issues.apache.org/jira/browse/HBASE-11646 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11646-V2.diff, HBASE-11646-V3.diff, HBASE-11646-V4.diff, HBASE-11646.diff In the updated MOB design however, admins can set CF level thresholds that would force cell values the threshold to use the MOB write path instead of the traditional path. There are two cases where mobs need to interact with this threshold 1) How do we handle the case when the threshold size is changed? 2) Today, you can bulkload hfiles that contain MOBs. These cells will work as normal inside hbase. Unfortunately the cells with MOBs in them will never benefit form the MOB write path. The proposal here is to modify compaction in mob enabled cf's such that the threshold value is honored with compactions. This handles case #1 -- elements that should be moved out of the normal hfiles get 'compacted' into refs and mob hfiles, and values that should be pulled into the cf get derefed and written out wholy in the compaction. For case #2, we can maintain the same behavior and compaction would move data into the mob writepath/lifecycle. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-11689: - Attachment: HBASE-11689.patch rebased to current master. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-11689: - Status: Patch Available (was: Open) Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-11689: - Status: Open (was: Patch Available) Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115115#comment-14115115
]
Hadoop QA commented on HBASE-11643:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665304/HBASE-11643-V17.diff
against trunk revision .
ATTACHMENT ID: 12665304
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10638//console
This message is automatically generated.
Read and write MOB in HBase
---
Key: HBASE-11643
URL: https://issues.apache.org/jira/browse/HBASE-11643
Project: HBase
Issue Type: Sub-task
Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff,
HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff,
HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff,
HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff,
HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff,
HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff
The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells
are saved in the MemStore, and flushed to the HFiles when necessary. For MOB,
the Cells are saved in the MemStore as well, but they're flushed to elsewhere
out of HBase in the format of HFiles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115119#comment-14115119
]
Hadoop QA commented on HBASE-11643:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665281/HBASE-11643-V16.diff
against trunk revision .
ATTACHMENT ID: 12665281
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 2.0.3) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestClassFinder
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10637//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10637//console
This message is automatically generated.
Read and write MOB in HBase
---
Key: HBASE-11643
URL: https://issues.apache.org/jira/browse/HBASE-11643
Project: HBase
Issue Type: Sub-task
Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff,
HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff,
HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff,
HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff,
HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff,
HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff
The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells
are saved in the MemStore, and flushed to the HFiles when necessary. For MOB,
the Cells are saved in the MemStore as well, but they're flushed to elsewhere
out of HBase in the format of HFiles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11851) RpcClient can try to close a connection not ready to close
[ https://issues.apache.org/jira/browse/HBASE-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115198#comment-14115198 ] Nicolas Liochon commented on HBASE-11851: - bq. but it seems a little odd to have a wait inside a synchronized here. Well if you want to stop the connection, you interrupt it as a connection is a thread. Obviously old fashion and pre-nio pattern :-/ Thanks for the review, I'm going to commit this. RpcClient can try to close a connection not ready to close -- Key: HBASE-11851 URL: https://issues.apache.org/jira/browse/HBASE-11851 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 Attachments: 11581.v1.patch see title -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115202#comment-14115202 ] Larry McCay commented on HBASE-11810: - Hey [~apurtell] - I'm not seeing this patch committed anywhere - should I? Since the jira is marked as fixed - I expected to see it in the source tree. Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[
https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115209#comment-14115209
]
Hadoop QA commented on HBASE-11689:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665307/HBASE-11689.patch
against trunk revision .
ATTACHMENT ID: 12665307
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.TestRegionRebalancing
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10639//console
This message is automatically generated.
Track meta in transition
Key: HBASE-11689
URL: https://issues.apache.org/jira/browse/HBASE-11689
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 1.0.0, 2.0.0
Reporter: Jimmy Xiang
Assignee: Andrey Stepachev
Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch,
HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch,
HBASE-11689.patch
With ZK-less region assignment, user regions in transition are tracked in
meta. We need a way to track meta in transition too.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11851) RpcClient can try to close a connection not ready to close
[ https://issues.apache.org/jira/browse/HBASE-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11851: Resolution: Fixed Fix Version/s: 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed, thanks for the review, Jon! RpcClient can try to close a connection not ready to close -- Key: HBASE-11851 URL: https://issues.apache.org/jira/browse/HBASE-11851 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 2.0.0 Attachments: 11581.v1.patch see title -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11821) [ImportTSV] Abstract labels tags creation into pluggable Interface
[ https://issues.apache.org/jira/browse/HBASE-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11821: --- Attachment: (was: HBASE-11821_V3.patch) [ImportTSV] Abstract labels tags creation into pluggable Interface -- Key: HBASE-11821 URL: https://issues.apache.org/jira/browse/HBASE-11821 Project: HBase Issue Type: Improvement Components: mapreduce, security Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 0.99.0, 2.0.0, 0.98.7 Attachments: HBASE-11821.patch HBASE-11553 abstracted out the VisibilityLabelService and using which one can go with String based labels tags instead of default ordinal based. In MapReduce side, ImportTSV also should support such an abstraction layer. This is applicable for HFileOutputFormat where mappers create Cells to be written to HFiles. Also every Mapper (custom written also) dealing with such low level HBase things (Creation of tags and Cells) is not a good idea. This issue plan to provide a public audience Facade class to facilitate such Cell creation so that Mappers can just make use of them. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11821) [ImportTSV] Abstract labels tags creation into pluggable Interface
[ https://issues.apache.org/jira/browse/HBASE-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11821: --- Attachment: HBASE-11821_V3.patch [ImportTSV] Abstract labels tags creation into pluggable Interface -- Key: HBASE-11821 URL: https://issues.apache.org/jira/browse/HBASE-11821 Project: HBase Issue Type: Improvement Components: mapreduce, security Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 0.99.0, 2.0.0, 0.98.7 Attachments: HBASE-11821.patch, HBASE-11821_V3.patch HBASE-11553 abstracted out the VisibilityLabelService and using which one can go with String based labels tags instead of default ordinal based. In MapReduce side, ImportTSV also should support such an abstraction layer. This is applicable for HFileOutputFormat where mappers create Cells to be written to HFiles. Also every Mapper (custom written also) dealing with such low level HBase things (Creation of tags and Cells) is not a good idea. This issue plan to provide a public audience Facade class to facilitate such Cell creation so that Mappers can just make use of them. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11643) Read and write MOB in HBase
[ https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11643: - Attachment: HBASE-11643-V18.diff Update the patch to delete the table directory where the mob files are saved. Read and write MOB in HBase --- Key: HBASE-11643 URL: https://issues.apache.org/jira/browse/HBASE-11643 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff, HBASE-11643-V18.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, the Cells are saved in the MemStore as well, but they're flushed to elsewhere out of HBase in the format of HFiles. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11860) Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
[ https://issues.apache.org/jira/browse/HBASE-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115266#comment-14115266 ] Sean Busbey commented on HBASE-11860: - Is your proposal that we adopt a position of always being able to upgrade to a released version from any supported version at the time of release? e.g. when presumably when 1.0 comes out 0.94 and 0.98 will still be around and you'd like users to be able to go directly to 1.0 from either (and rolling from 0.98?). Similarly current 0.94 users would be able to go directly to 0.98? Is there already a dev@ discussion around this? Since our [upgrade docs|http://hbase.apache.org/book.html#upgrading] expressly call out not supporting this kind of upgrades it's probably worth preemptively asking for concerns from the wider audience there. Restore UpgradeTo in order to upgrade a HBase cluster to the latest release --- Key: HBASE-11860 URL: https://issues.apache.org/jira/browse/HBASE-11860 Project: HBase Issue Type: Improvement Components: HFile, master Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Fix For: 0.99.0, 2.0.0 Users should have the option to upgrade to a newer release of HBase without performing a transitive upgrade to HBase 0.96 (EOL'd) or HBase 0.98 and then HBase 1.x or 2.x In HBASE-11611 UpgradeTo96 was removed and it won't be possible to migrate a HBase 0.94 cluster to HBase 1.x or 2.x without the extra step. Initially we can restore part of the functionality of the tool supporting migration to namespaces and report for the existence of any HFileV1 file and ZKDataMigrator could be safely removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[ https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115276#comment-14115276 ] Ted Yu commented on HBASE-11857: Codec is referenced in: phoenix-core/src/it/java/org/apache/hadoop/hbase/regionserver/wal/ReadWriteKeyValuesWithCodecIT.java ReaderBase is not referenced in phoenix. Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) Key: HBASE-11857 URL: https://issues.apache.org/jira/browse/HBASE-11857 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Blocker Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: 11857-trunk-v1.txt ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) were removed when adding Cell codec to WAL header. Some downstream projects, such as Phoenix, depend on these methods. This JIRA is to restore these methods. Thanks to Jeffrey who found this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[ https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-11857: --- Attachment: 11857-trunk-v2.txt Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) Key: HBASE-11857 URL: https://issues.apache.org/jira/browse/HBASE-11857 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Blocker Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: 11857-trunk-v1.txt, 11857-trunk-v2.txt ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) were removed when adding Cell codec to WAL header. Some downstream projects, such as Phoenix, depend on these methods. This JIRA is to restore these methods. Thanks to Jeffrey who found this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11821) [ImportTSV] Abstract labels tags creation into pluggable Interface
[
https://issues.apache.org/jira/browse/HBASE-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115312#comment-14115312
]
Hadoop QA commented on HBASE-11821:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665334/HBASE-11821_V3.patch
against trunk revision .
ATTACHMENT ID: 12665334
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10640//console
This message is automatically generated.
[ImportTSV] Abstract labels tags creation into pluggable Interface
--
Key: HBASE-11821
URL: https://issues.apache.org/jira/browse/HBASE-11821
Project: HBase
Issue Type: Improvement
Components: mapreduce, security
Affects Versions: 0.98.0
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.99.0, 2.0.0, 0.98.7
Attachments: HBASE-11821.patch, HBASE-11821_V3.patch
HBASE-11553 abstracted out the VisibilityLabelService and using which one can
go with String based labels tags instead of default ordinal based. In
MapReduce side, ImportTSV also should support such an abstraction layer.
This is applicable for HFileOutputFormat where mappers create Cells to be
written to HFiles.
Also every Mapper (custom written also) dealing with such low level HBase
things (Creation of tags and Cells) is not a good idea. This issue plan to
provide a public audience Facade class to facilitate such Cell creation so
that Mappers can just make use of them.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115314#comment-14115314
]
Hadoop QA commented on HBASE-11643:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665337/HBASE-11643-V18.diff
against trunk revision .
ATTACHMENT ID: 12665337
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 2.0.3) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestClassFinder
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10641//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10641//console
This message is automatically generated.
Read and write MOB in HBase
---
Key: HBASE-11643
URL: https://issues.apache.org/jira/browse/HBASE-11643
Project: HBase
Issue Type: Sub-task
Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff,
HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff,
HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff,
HBASE-11643-V18.diff, HBASE-11643-V2.diff, HBASE-11643-V3.diff,
HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff,
HBASE-11643-V7.diff, HBASE-11643-V8.diff, HBASE-11643-V9.diff,
HBase-11643.diff
The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells
are saved in the MemStore, and flushed to the HFiles when necessary. For MOB,
the Cells are saved in the MemStore as well, but they're flushed to elsewhere
out of HBase in the format of HFiles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11851) RpcClient can try to close a connection not ready to close
[ https://issues.apache.org/jira/browse/HBASE-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115320#comment-14115320 ] Hudson commented on HBASE-11851: FAILURE: Integrated in HBase-TRUNK #5445 (See [https://builds.apache.org/job/HBase-TRUNK/5445/]) HBASE-11851 RpcClient can try to close a connection not ready to close (nkeywal: rev 8c0e5dca75955ad948d706d8f4f849615501eaeb) * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java RpcClient can try to close a connection not ready to close -- Key: HBASE-11851 URL: https://issues.apache.org/jira/browse/HBASE-11851 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 2.0.0 Attachments: 11581.v1.patch see title -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115333#comment-14115333 ] Jimmy Xiang commented on HBASE-11689: - I am making some change to the patch. I will upload the revised patch shortly. Thanks. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115339#comment-14115339 ] Andrew Purtell commented on HBASE-11810: Should be there now Larry, please check. See https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115340#comment-14115340 ] Andrey Stepachev commented on HBASE-11689: -- Thank you [~jxiang]. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[ https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-11857: --- Attachment: 11857-trunk-v3.txt Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) Key: HBASE-11857 URL: https://issues.apache.org/jira/browse/HBASE-11857 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Blocker Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: 11857-trunk-v1.txt, 11857-trunk-v2.txt, 11857-trunk-v3.txt ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) were removed when adding Cell codec to WAL header. Some downstream projects, such as Phoenix, depend on these methods. This JIRA is to restore these methods. Thanks to Jeffrey who found this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[ https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115345#comment-14115345 ] Andrew Purtell commented on HBASE-11857: bq. ReaderBase is not referenced in phoenix. Then why is it necessary to restore ReaderBase#initAfterCompression as the title of this JIRA says? I agree with Anoop, classes used by the interfaces that Phoenix is using and related abstract classes should all be tagged LimitedPrivate(COPROC,PHOENIX) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) Key: HBASE-11857 URL: https://issues.apache.org/jira/browse/HBASE-11857 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Blocker Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: 11857-trunk-v1.txt, 11857-trunk-v2.txt ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext) were removed when adding Cell codec to WAL header. Some downstream projects, such as Phoenix, depend on these methods. This JIRA is to restore these methods. Thanks to Jeffrey who found this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115358#comment-14115358
]
Esteban Gutierrez commented on HBASE-11726:
---
Lastest patch removes some garbage from the patch related to NamespaceJanitor
removal.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HBASE-11726:
--
Attachment: HBASE-11726.v2.diff
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11851) RpcClient can try to close a connection not ready to close
[ https://issues.apache.org/jira/browse/HBASE-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115359#comment-14115359 ] Hudson commented on HBASE-11851: FAILURE: Integrated in HBase-1.0 #137 (See [https://builds.apache.org/job/HBase-1.0/137/]) HBASE-11851 RpcClient can try to close a connection not ready to close (nkeywal: rev d12ad173817da94b2a11899686fa52d5b866e679) * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java RpcClient can try to close a connection not ready to close -- Key: HBASE-11851 URL: https://issues.apache.org/jira/browse/HBASE-11851 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 2.0.0 Attachments: 11581.v1.patch see title -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[
https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115370#comment-14115370
]
Hadoop QA commented on HBASE-11857:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665345/11857-trunk-v2.txt
against trunk revision .
ATTACHMENT ID: 12665345
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10642//console
This message is automatically generated.
Restore ReaderBase.initAfterCompression() and
WALCellCodec.create(Configuration, CompressionContext)
Key: HBASE-11857
URL: https://issues.apache.org/jira/browse/HBASE-11857
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 11857-trunk-v1.txt, 11857-trunk-v2.txt,
11857-trunk-v3.txt
ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration,
CompressionContext) were removed when adding Cell codec to WAL header.
Some downstream projects, such as Phoenix, depend on these methods.
This JIRA is to restore these methods.
Thanks to Jeffrey who found this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115373#comment-14115373 ] Larry McCay commented on HBASE-11810: - Cool - it's there now. Is there a different backend process for the actual commit in hbase than in hadoop? On Fri, Aug 29, 2014 at 11:36 AM, Andrew Purtell (JIRA) j...@apache.org Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11860) Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
[ https://issues.apache.org/jira/browse/HBASE-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115379#comment-14115379 ] stack commented on HBASE-11860: --- First off, what Sean says ...our upgrade docs expressly call out not supporting this kind of upgrades That said, it seems going 0.94 to 1.0 wouldn't take much work. If it is just a case of leaving around some deprecated APIs and restoring old scripts removed in zeal, I'd be game for testing and making sure it basically works. Restore UpgradeTo in order to upgrade a HBase cluster to the latest release --- Key: HBASE-11860 URL: https://issues.apache.org/jira/browse/HBASE-11860 Project: HBase Issue Type: Improvement Components: HFile, master Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Fix For: 0.99.0, 2.0.0 Users should have the option to upgrade to a newer release of HBase without performing a transitive upgrade to HBase 0.96 (EOL'd) or HBase 0.98 and then HBase 1.x or 2.x In HBASE-11611 UpgradeTo96 was removed and it won't be possible to migrate a HBase 0.94 cluster to HBase 1.x or 2.x without the extra step. Initially we can restore part of the functionality of the tool supporting migration to namespaces and report for the existence of any HFileV1 file and ZKDataMigrator could be safely removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11689: Attachment: hbase-11689-revised.patch Attached the revised the patch. [~octo47], could you please take a look? If it is ok with you, we can get it into the master branch. For 1.0, it needs some change since 1.0 still uses ZK for assignment by default. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, hbase-11689-revised.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115418#comment-14115418 ] Hudson commented on HBASE-11810: FAILURE: Integrated in HBase-TRUNK #5446 (See [https://builds.apache.org/job/HBase-TRUNK/5446/]) HBASE-11810 Access SSL Passwords through Credential Provider API (Larry McCay) (apurtell: rev e5a5e968f16bdeae3b2b16657c2c7c54ea025579) * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11860) Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
[
https://issues.apache.org/jira/browse/HBASE-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115420#comment-14115420
]
Esteban Gutierrez commented on HBASE-11860:
---
[~busbey]:
{quote}
Is your proposal that we adopt a position of always being able to upgrade to a
released version from any supported version at the time of release?
{quote}
Not across all versions but at least the set of versions actively in use. I've
seen that HBase production clusters life is around 2 years with the same
version and sometimes do major upgrades skipping a couple of intermediate
versions, e.g 0.20 - 0.94, 0.90 - 0.94, 0.94 - 0.98. At least we should
support upgrades from previous releases to HBase 1.x.
{quote}
Since our upgrade docs expressly call out not supporting this kind of
upgrades
{quote}
I think if we have the functionality to migrate a cluster from 0.90 to 0.98
then we should carry that functionality at least to HBase 1.x. Since 0.96 has
been EOL'd I think it should be fine to keep that functionality a little
longer.
[~stack]
{quote}
First off, what Sean says ...our upgrade docs expressly call out not
supporting this kind of upgrades That said, it seems going 0.94 to 1.0
wouldn't take much work. If it is just a case of leaving around some deprecated
APIs and restoring old scripts removed in zeal, I'd be game for testing and
making sure it basically works.
{quote}
Thats right, basically the old UpgradeTo96 works and it could be required
that the upgrade should be done first to use the old ZK based assignment so
existing znodes could be migrated, then users can opt to use the zk-less
assignment. If using the zk-less assignment we only need to migrate the HBase
filesystem to the new layout.
P.S. +1 I think we can bring this discussion into the dev@ mailing list.
Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
---
Key: HBASE-11860
URL: https://issues.apache.org/jira/browse/HBASE-11860
Project: HBase
Issue Type: Improvement
Components: HFile, master
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Fix For: 0.99.0, 2.0.0
Users should have the option to upgrade to a newer release of HBase without
performing a transitive upgrade to HBase 0.96 (EOL'd) or HBase 0.98 and then
HBase 1.x or 2.x
In HBASE-11611 UpgradeTo96 was removed and it won't be possible to migrate a
HBase 0.94 cluster to HBase 1.x or 2.x without the extra step.
Initially we can restore part of the functionality of the tool supporting
migration to namespaces and report for the existence of any HFileV1 file and
ZKDataMigrator could be safely removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115440#comment-14115440
]
Jimmy Xiang commented on HBASE-11726:
-
+1
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11788) hbase is not deleting the cell when a Put with a KeyValue, KeyValue.Type.Delete is submitted
[
https://issues.apache.org/jira/browse/HBASE-11788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115457#comment-14115457
]
Jimmy Xiang commented on HBASE-11788:
-
It's hacky. Probably it is not expected the API to be used this way. Should we
ignore the ts if it is set to LATEST_TIMESTAMP?
hbase is not deleting the cell when a Put with a KeyValue,
KeyValue.Type.Delete is submitted
Key: HBASE-11788
URL: https://issues.apache.org/jira/browse/HBASE-11788
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.96.1.1, 0.98.5, 2.0.0
Environment: Cloudera CDH 5.1.x
Reporter: Cristian Armaselu
Assignee: Srikanth Srungarapu
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11788-master.patch, HBASE-11788-master_v2.patch,
TestPutWithDelete.java
Code executed:
{code}
@Test
public void testHbasePutDeleteCell() throws Exception {
TableName tableName = TableName.valueOf(my_test);
Configuration configuration = HBaseConfiguration.create();
HTableInterface table = new HTable(configuration, tableName);
final String rowKey = 12345;
final byte[] familly = Bytes.toBytes(default);
// put one row
Put put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(familly, Bytes.toBytes(C), Bytes.toBytes(c));
table.put(put);
// get row back and assert the values
Get get = new Get(Bytes.toBytes(rowKey));
Result result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column B value should be b);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(C))).equals(c), Column C value should be c);
// put the same row again with C column deleted
put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(new KeyValue(Bytes.toBytes(rowKey), familly,
Bytes.toBytes(C), HConstants.LATEST_TIMESTAMP, KeyValue.Type.DeleteColumn));
table.put(put);
// get row back and assert the values
get = new Get(Bytes.toBytes(rowKey));
result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column A value should be b);
Assert.isTrue(result.getValue(familly, Bytes.toBytes(C)) == null,
Column C should not exists);
}
{code}
This assertion fails, the cell is not deleted but rather the value is empty:
{code}
hbase(main):029:0 scan 'my_test'
ROW COLUMN+CELL
12345column=default:A,
timestamp=1408473082290, value=a
12345column=default:B,
timestamp=1408473082290, value=b
12345column=default:C,
timestamp=1408473082290, value=
{code}
This behavior is different than previous 4.8.x Cloudera version and is
currently corrupting all hive queries involving is null or is not null
operators on the columns mapped to hbase
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115463#comment-14115463
]
stack commented on HBASE-11726:
---
Patch looks good [~esteban] Thanks for fixing this nasty. Have you tried
starting a 0.96+ over a 0.94 w/o migration to see if above works? We should
put this in 0.98+ or should I put it into 0.96 too? Make a release even?
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[ https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115480#comment-14115480 ] stack commented on HBASE-11777: --- +1 on patch. On commit, mind renaming SequenceNumber to SequenceId and SequenceNumber#getSequenceNumber to getSequenceId so they align with the new interface you are introducing here? Thanks Anoop. Find a way to use KV.setSequenceId() on Cells on the server-side read path -- Key: HBASE-11777 URL: https://issues.apache.org/jira/browse/HBASE-11777 Project: HBase Issue Type: Improvement Affects Versions: 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: CellWithSequenceNumber.java, HBASE-11777.patch, HBASE-11777_V2.patch Over in HBASE-11591 there was a need to set the sequenceId of the HFile to the bulk loaded KVs. Since we are trying to use the concept of Cells in the read path if we need to use setSequenceId(), then the Cell has to be converted to KV and only KeyValue impl has the accessor setSequenceId(). [~anoop.hbase] suggested if we can use a Server side impl of Cell and have these accessors in them. This JIRA aims to solve this and see the related code changes that needs to be carried out for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115485#comment-14115485
]
Esteban Gutierrez commented on HBASE-11726:
---
I tried with trunk and looks good, the hbase.version file is not migrated to PB
anymore. And yeah, we should have it in 0.98 to avoid that issue. I don't see
need for 0.96 since it has been EOL'd.
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11788) hbase is not deleting the cell when a Put with a KeyValue, KeyValue.Type.Delete is submitted
[
https://issues.apache.org/jira/browse/HBASE-11788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115497#comment-14115497
]
Jimmy Xiang commented on HBASE-11788:
-
bq. Once you do such a delete, you will not be able to Put a same rk:cf:q cell.
With Delete, you can specify the ts to be LATEST_TIMESTAMP, right? After such a
delete, will you be able to do Put to the same cell?
hbase is not deleting the cell when a Put with a KeyValue,
KeyValue.Type.Delete is submitted
Key: HBASE-11788
URL: https://issues.apache.org/jira/browse/HBASE-11788
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.96.1.1, 0.98.5, 2.0.0
Environment: Cloudera CDH 5.1.x
Reporter: Cristian Armaselu
Assignee: Srikanth Srungarapu
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11788-master.patch, HBASE-11788-master_v2.patch,
TestPutWithDelete.java
Code executed:
{code}
@Test
public void testHbasePutDeleteCell() throws Exception {
TableName tableName = TableName.valueOf(my_test);
Configuration configuration = HBaseConfiguration.create();
HTableInterface table = new HTable(configuration, tableName);
final String rowKey = 12345;
final byte[] familly = Bytes.toBytes(default);
// put one row
Put put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(familly, Bytes.toBytes(C), Bytes.toBytes(c));
table.put(put);
// get row back and assert the values
Get get = new Get(Bytes.toBytes(rowKey));
Result result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column B value should be b);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(C))).equals(c), Column C value should be c);
// put the same row again with C column deleted
put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(new KeyValue(Bytes.toBytes(rowKey), familly,
Bytes.toBytes(C), HConstants.LATEST_TIMESTAMP, KeyValue.Type.DeleteColumn));
table.put(put);
// get row back and assert the values
get = new Get(Bytes.toBytes(rowKey));
result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column A value should be b);
Assert.isTrue(result.getValue(familly, Bytes.toBytes(C)) == null,
Column C should not exists);
}
{code}
This assertion fails, the cell is not deleted but rather the value is empty:
{code}
hbase(main):029:0 scan 'my_test'
ROW COLUMN+CELL
12345column=default:A,
timestamp=1408473082290, value=a
12345column=default:B,
timestamp=1408473082290, value=b
12345column=default:C,
timestamp=1408473082290, value=
{code}
This behavior is different than previous 4.8.x Cloudera version and is
currently corrupting all hive queries involving is null or is not null
operators on the columns mapped to hbase
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[ https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115505#comment-14115505 ] Anoop Sam John commented on HBASE-11777: I see. You were talking about the existing SequenceNumber interface . Sure I will change this on commit. Thanks boss. Find a way to use KV.setSequenceId() on Cells on the server-side read path -- Key: HBASE-11777 URL: https://issues.apache.org/jira/browse/HBASE-11777 Project: HBase Issue Type: Improvement Affects Versions: 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: CellWithSequenceNumber.java, HBASE-11777.patch, HBASE-11777_V2.patch Over in HBASE-11591 there was a need to set the sequenceId of the HFile to the bulk loaded KVs. Since we are trying to use the concept of Cells in the read path if we need to use setSequenceId(), then the Cell has to be converted to KV and only KeyValue impl has the accessor setSequenceId(). [~anoop.hbase] suggested if we can use a Server side impl of Cell and have these accessors in them. This JIRA aims to solve this and see the related code changes that needs to be carried out for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115510#comment-14115510 ] Hudson commented on HBASE-11810: SUCCESS: Integrated in HBase-0.98 #484 (See [https://builds.apache.org/job/HBase-0.98/484/]) HBASE-11810 Access SSL Passwords through Credential Provider API (Larry McCay) (apurtell: rev c970a85cae355782e1a60bdec36f8340d8e1e422) * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11859) 'hadoop jar' references in documentation should mention hbase-server.jar, not hbase.jar
[ https://issues.apache.org/jira/browse/HBASE-11859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115512#comment-14115512 ] Jonathan Hsieh commented on HBASE-11859: These failures are unrelated to this patch. 'hadoop jar' references in documentation should mention hbase-server.jar, not hbase.jar --- Key: HBASE-11859 URL: https://issues.apache.org/jira/browse/HBASE-11859 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.99.0, 2.0.0, 0.98.6 Reporter: Stephen Veiss Assignee: Stephen Veiss Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11859.branch1.patch, HBASE-11859.master.patch, HBASE-11859.patch There are various org.apache.hadoop.util.Tool implementations mentioned in the documentation as being run with hadoop jar hbase-VERSION.jar toolname. These classes now live in in the hbase-server module, so that jar name should be hbase-server-VERSION.jar. The same applies to the documentation on running MapReduce jobs against HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11859) 'hadoop jar' references in documentation should mention hbase-server.jar, not hbase.jar
[ https://issues.apache.org/jira/browse/HBASE-11859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11859: --- Resolution: Fixed Fix Version/s: 2.0.0 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) 'hadoop jar' references in documentation should mention hbase-server.jar, not hbase.jar --- Key: HBASE-11859 URL: https://issues.apache.org/jira/browse/HBASE-11859 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.99.0, 2.0.0, 0.98.6 Reporter: Stephen Veiss Assignee: Stephen Veiss Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11859.branch1.patch, HBASE-11859.master.patch, HBASE-11859.patch There are various org.apache.hadoop.util.Tool implementations mentioned in the documentation as being run with hadoop jar hbase-VERSION.jar toolname. These classes now live in in the hbase-server module, so that jar name should be hbase-server-VERSION.jar. The same applies to the documentation on running MapReduce jobs against HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115524#comment-14115524 ] stack commented on HBASE-11165: --- bq. ..by the way, could somebody estimate the distribution or read vs writes to meta table, in terms of requests per second/networking traffic/disk access? [~toffer] or [~virag] Do you have any numbers from your experiments? bq. I believe there are 2 dimensions here, right? There is a more immediate 3rd dimension/option and that is what we have now where we have a single master and if it fails, backup assumes its role. For your dimension 1., (also known as HBASE-10296 Replace ZK with a consensus lib(paxos,zab or raft running within master processes to provide better master failover performance and state consistency), it would be a nice-to-have but if we can, lets try and avoid our having to have a HA master. As long as the master comes back inside some reasonable window and the cluster can keep on chugging while its figuring out its recovery, this would be a simpler deploy than one that needs a HA master mini-cluster. Multi-master would help scale reads but as you note, doesn't help if one massive meta region and we want to cache it. We'd go to partitioned masters, as I see it, if we can't make one master do. bq. The viable combined solution ... I suggest we look at single master with split meta served by regionservers first? Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115526#comment-14115526 ] Jonathan Hsieh edited comment on HBASE-11339 at 8/29/14 5:24 PM: - [~jiajia], new version of docs look good, I think it is done for now unless we make changes to it. nits: I found there are two typos, provinding \- providing and handers\-handlers.Don't worry about fixing this for now -- we'll have [~misty] convert them into a chapter or section in the ref guide. Also, in the future, please do not delete attachments -- just provide a new version with a v2 or some think like that so we can keep track of the evolution. was (Author: jmhsieh): [~jiajia], new version of docs look good, I think it is done for now unless we make changes to it. nits: I found there are two typos, provinding - providing and handers-handlers.Don't worry about fixing this for now -- we'll have [~misty] convert them into a chapter or section in the ref guide. Also, in the future, please do not delete attachments -- just provide a new version with a v2 or some think like that so we can keep track of the evolution. HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: Umbrella Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB Design.pdf, MOB user guide.docx, hbase-11339-in-dev.patch It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115526#comment-14115526 ] Jonathan Hsieh commented on HBASE-11339: [~jiajia], new version of docs look good, I think it is done for now unless we make changes to it. nits: I found there are two typos, provinding - providing and handers-handlers.Don't worry about fixing this for now -- we'll have [~misty] convert them into a chapter or section in the ref guide. Also, in the future, please do not delete attachments -- just provide a new version with a v2 or some think like that so we can keep track of the evolution. HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: Umbrella Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB Design.pdf, MOB user guide.docx, hbase-11339-in-dev.patch It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11861) Native MOB Compaction mechanisms.
Jonathan Hsieh created HBASE-11861: -- Summary: Native MOB Compaction mechanisms. Key: HBASE-11861 URL: https://issues.apache.org/jira/browse/HBASE-11861 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Currenlty, the first cut of mob will have external processes to age off old mob data (the ttl cleaner), and to compact away deleted or over written data (the sweep tool). From an operational point of view, having two external tools, especially one that relies on MapReduce is undesirable. In this issue we'll tackle integrating these into hbase without requiring external processes. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11861) Native MOB Compaction mechanisms.
[ https://issues.apache.org/jira/browse/HBASE-11861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115537#comment-14115537 ] Jonathan Hsieh commented on HBASE-11861: Two straw man suggestions. 1) Integrate the TTL cleaner as a Chore that the master runs periodically (once every 24 hours by default?) 2) Explore ideas to enable a per mob hfile sweep (traverse back to natural hbase row do some sort of bulk load with all the new pointers) so that we can compact without using mr. Native MOB Compaction mechanisms. - Key: HBASE-11861 URL: https://issues.apache.org/jira/browse/HBASE-11861 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Currenlty, the first cut of mob will have external processes to age off old mob data (the ttl cleaner), and to compact away deleted or over written data (the sweep tool). From an operational point of view, having two external tools, especially one that relies on MapReduce is undesirable. In this issue we'll tackle integrating these into hbase without requiring external processes. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11788) hbase is not deleting the cell when a Put with a KeyValue, KeyValue.Type.Delete is submitted
[
https://issues.apache.org/jira/browse/HBASE-11788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115548#comment-14115548
]
Anoop Sam John commented on HBASE-11788:
I mean the delete version case. But seems not much of issue like mark *all*
future Puts. Becuase we treat the cell as Put type and update the latest TS
with current RS time. So only issue is it might not delete the version
intended. But might delete a future version (If an immediate put or put with
same TS)
hbase is not deleting the cell when a Put with a KeyValue,
KeyValue.Type.Delete is submitted
Key: HBASE-11788
URL: https://issues.apache.org/jira/browse/HBASE-11788
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.96.1.1, 0.98.5, 2.0.0
Environment: Cloudera CDH 5.1.x
Reporter: Cristian Armaselu
Assignee: Srikanth Srungarapu
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11788-master.patch, HBASE-11788-master_v2.patch,
TestPutWithDelete.java
Code executed:
{code}
@Test
public void testHbasePutDeleteCell() throws Exception {
TableName tableName = TableName.valueOf(my_test);
Configuration configuration = HBaseConfiguration.create();
HTableInterface table = new HTable(configuration, tableName);
final String rowKey = 12345;
final byte[] familly = Bytes.toBytes(default);
// put one row
Put put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(familly, Bytes.toBytes(C), Bytes.toBytes(c));
table.put(put);
// get row back and assert the values
Get get = new Get(Bytes.toBytes(rowKey));
Result result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column B value should be b);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(C))).equals(c), Column C value should be c);
// put the same row again with C column deleted
put = new Put(Bytes.toBytes(rowKey));
put.add(familly, Bytes.toBytes(A), Bytes.toBytes(a));
put.add(familly, Bytes.toBytes(B), Bytes.toBytes(b));
put.add(new KeyValue(Bytes.toBytes(rowKey), familly,
Bytes.toBytes(C), HConstants.LATEST_TIMESTAMP, KeyValue.Type.DeleteColumn));
table.put(put);
// get row back and assert the values
get = new Get(Bytes.toBytes(rowKey));
result = table.get(get);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(A))).equals(a), Column A value should be a);
Assert.isTrue(Bytes.toString(result.getValue(familly,
Bytes.toBytes(B))).equals(b), Column A value should be b);
Assert.isTrue(result.getValue(familly, Bytes.toBytes(C)) == null,
Column C should not exists);
}
{code}
This assertion fails, the cell is not deleted but rather the value is empty:
{code}
hbase(main):029:0 scan 'my_test'
ROW COLUMN+CELL
12345column=default:A,
timestamp=1408473082290, value=a
12345column=default:B,
timestamp=1408473082290, value=b
12345column=default:C,
timestamp=1408473082290, value=
{code}
This behavior is different than previous 4.8.x Cloudera version and is
currently corrupting all hive queries involving is null or is not null
operators on the columns mapped to hbase
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[ https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11777: --- Attachment: HBASE-11777_V3.patch Let QA run on V3 Find a way to use KV.setSequenceId() on Cells on the server-side read path -- Key: HBASE-11777 URL: https://issues.apache.org/jira/browse/HBASE-11777 Project: HBase Issue Type: Improvement Affects Versions: 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: CellWithSequenceNumber.java, HBASE-11777.patch, HBASE-11777_V2.patch, HBASE-11777_V3.patch Over in HBASE-11591 there was a need to set the sequenceId of the HFile to the bulk loaded KVs. Since we are trying to use the concept of Cells in the read path if we need to use setSequenceId(), then the Cell has to be converted to KV and only KeyValue impl has the accessor setSequenceId(). [~anoop.hbase] suggested if we can use a Server side impl of Cell and have these accessors in them. This JIRA aims to solve this and see the related code changes that needs to be carried out for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115551#comment-14115551 ] Hudson commented on HBASE-11810: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #457 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/457/]) HBASE-11810 Access SSL Passwords through Credential Provider API (Larry McCay) (apurtell: rev c970a85cae355782e1a60bdec36f8340d8e1e422) * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11855) HBase handbook chapter 18.9 out of date
[ https://issues.apache.org/jira/browse/HBASE-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115560#comment-14115560 ] Michael Tauscher commented on HBASE-11855: -- Thanks [~misty] and [~busbey] again, consider me unofficially assigned to this - I've got a patch coming today. HBase handbook chapter 18.9 out of date --- Key: HBASE-11855 URL: https://issues.apache.org/jira/browse/HBASE-11855 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 2.0.0 Reporter: Michael Tauscher Priority: Minor Recently [~posix4e] committed a change [HBASE-4955|https://issues.apache.org/jira/browse/HBASE-4955] renaming Dsurefire.*PartThreadCount to Dsurefire.*PartForkCount for the 2.0 version. I'm excited to help with documentation and the webpage going forward. Can someone help mentor me with the process? Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11851) RpcClient can try to close a connection not ready to close
[ https://issues.apache.org/jira/browse/HBASE-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115557#comment-14115557 ] Jonathan Hsieh commented on HBASE-11851: [~nkeywal], it would be nice to have a comment with the interrupt pattern near the wait -- justifying bad smells with comments really helps. RpcClient can try to close a connection not ready to close -- Key: HBASE-11851 URL: https://issues.apache.org/jira/browse/HBASE-11851 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 2.0.0 Attachments: 11581.v1.patch see title -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115558#comment-14115558 ] Mikhail Antonov commented on HBASE-11165: - Oops, sorry, instead of For multi-masters (replicated master) the objection I believe is, I meant ..objective. Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115562#comment-14115562 ] Jimmy Xiang commented on HBASE-11165: - One meta region should be good enough for most of clusters. If there is just one meta region, it's better to put it together with the master to avoid possible complications. Additional meta regions should be optional. It should be simple to add new meta regions. Ideally, the same code path should be used no matter how many meta regions there are. Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115582#comment-14115582 ] Mikhail Antonov commented on HBASE-11165: - Totally agree that for most clusters, 1 meta region should suffice, and that if there's one region, it's easier to bind it to master (no rpcs, less complicated coordination and failure scenarios). Though, for clusters where 1 meta regions is required, that would need to be served on other machines using RPC, right? I mean, seems tricky to make it be the same code path? Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11810) Access SSL Passwords through Credential Provider API
[ https://issues.apache.org/jira/browse/HBASE-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115595#comment-14115595 ] Hudson commented on HBASE-11810: SUCCESS: Integrated in HBase-1.0 #138 (See [https://builds.apache.org/job/HBase-1.0/138/]) HBASE-11810 Access SSL Passwords through Credential Provider API (Larry McCay) (apurtell: rev 51cf5c359b47a43780f86e5302128dd5bc324620) * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java * hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java Access SSL Passwords through Credential Provider API Key: HBASE-11810 URL: https://issues.apache.org/jira/browse/HBASE-11810 Project: HBase Issue Type: Improvement Components: security Reporter: Larry McCay Assignee: Larry McCay Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11810-2.patch, HBASE_11810.patch HADOOP-10607 introduced the credential provider API for allowing passwords and other sensitive configuration items to be stored in an external provider. RESTServer is accessing passwords stored in clear text in Configuration through the standard get() method. By using the new Configuration.getPassword method instead, the credential provider API will be checked first then fall back to clear text - when allowed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[ https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115607#comment-14115607 ] stack commented on HBASE-11777: --- +1 Thanks [~anoop.hbase] Find a way to use KV.setSequenceId() on Cells on the server-side read path -- Key: HBASE-11777 URL: https://issues.apache.org/jira/browse/HBASE-11777 Project: HBase Issue Type: Improvement Affects Versions: 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: CellWithSequenceNumber.java, HBASE-11777.patch, HBASE-11777_V2.patch, HBASE-11777_V3.patch Over in HBASE-11591 there was a need to set the sequenceId of the HFile to the bulk loaded KVs. Since we are trying to use the concept of Cells in the read path if we need to use setSequenceId(), then the Cell has to be converted to KV and only KeyValue impl has the accessor setSequenceId(). [~anoop.hbase] suggested if we can use a Server side impl of Cell and have these accessors in them. This JIRA aims to solve this and see the related code changes that needs to be carried out for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11855) HBase handbook chapter 18.9 out of date
[ https://issues.apache.org/jira/browse/HBASE-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11855: -- Assignee: Michael Tauscher HBase handbook chapter 18.9 out of date --- Key: HBASE-11855 URL: https://issues.apache.org/jira/browse/HBASE-11855 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 2.0.0 Reporter: Michael Tauscher Assignee: Michael Tauscher Priority: Minor Recently [~posix4e] committed a change [HBASE-4955|https://issues.apache.org/jira/browse/HBASE-4955] renaming Dsurefire.*PartThreadCount to Dsurefire.*PartForkCount for the 2.0 version. I'm excited to help with documentation and the webpage going forward. Can someone help mentor me with the process? Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[
https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115610#comment-14115610
]
Hadoop QA commented on HBASE-11689:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665365/hbase-11689-revised.patch
against trunk revision .
ATTACHMENT ID: 12665365
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10645//console
This message is automatically generated.
Track meta in transition
Key: HBASE-11689
URL: https://issues.apache.org/jira/browse/HBASE-11689
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 1.0.0, 2.0.0
Reporter: Jimmy Xiang
Assignee: Andrey Stepachev
Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch,
HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch,
HBASE-11689.patch, hbase-11689-revised.patch
With ZK-less region assignment, user regions in transition are tracked in
meta. We need a way to track meta in transition too.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11726:
--
Resolution: Fixed
Fix Version/s: 0.98.7
2.0.0
0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed 0.98+. Thanks Esteban (And Srikanth for review)
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.7
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115615#comment-14115615 ] Jimmy Xiang commented on HBASE-11689: - Looking the failed tests. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, hbase-11689-revised.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11837) Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no local data
[
https://issues.apache.org/jira/browse/HBASE-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-11837:
--
Resolution: Fixed
Fix Version/s: 0.98.7
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks [~andrew.purt...@gmail.com] and [~anoop.hbase] for reviews! I've
integrated the fix into 0.98 branch and there is no need for trunk branch.
Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no
local data
---
Key: HBASE-11837
URL: https://issues.apache.org/jira/browse/HBASE-11837
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.98.4
Reporter: James Taylor
Assignee: Jeffrey Zhong
Fix For: 0.98.7
Attachments: HBASE-11837.patch, PHOENIX-1208-torepro.patch
Might be fixed as a byproduct of the fix for HBASE-11766.
Here's what I'm seeing:
- From an endpoint coprocessor on SYSTEM.CATALOG table
- Rows exist in the table, but are all in a single region
- Scan is a new, empty scan created like this: new Scan()
- The following scanner *does* return data:
{code}
RegionScanner scanner = region.getScanner(scan);
{code}
- The following scanner *does not* return any data:
{code}
HTableInterface hTable = env.getTable(region.getTableDesc().getTableName());
{code}
- The following scanner *does* return data:
{code}
HTablePool pool = new HTablePool (env.getConfiguration(),1);
HTableInterface hTable = pool.getTable(SYSTEM.CATALOG);
{code}
[~jeffreyz], [~apurtell]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11826) Split each tableOrRegionName admin methods into two targetted methods
[
https://issues.apache.org/jira/browse/HBASE-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11826:
--
Attachment: HBASE_11826_v2.patch
+1 on patch. Nice. @enis going to commit to branch-1 in a day or so unless
objection ([~ndimiduk] you cool w/ dat?)
Split each tableOrRegionName admin methods into two targetted methods
-
Key: HBASE-11826
URL: https://issues.apache.org/jira/browse/HBASE-11826
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
Attachments: HBASE_11826.patch, HBASE_11826_v2.patch,
HBASE_11826_v2.patch
Purpose of this is two implement [~enis]'s suggestion to strongly type the
methods that take tableOrRegionName as an argument.
For instance:
{code}
void compact(final String tableNameOrRegionName)
void compact(final byte[] tableNameOrRegionName)
{code}
becomes
{code}
@Deprecated
void compact(final String tableNameOrRegionName)
@Deprecated
void compact(final byte[] tableNameOrRegionName)
void compact(TableName table)
void compactRegion(final byte[] regionName)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11857) Restore ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration, CompressionContext)
[
https://issues.apache.org/jira/browse/HBASE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115633#comment-14115633
]
Hadoop QA commented on HBASE-11857:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665358/11857-trunk-v3.txt
against trunk revision .
ATTACHMENT ID: 12665358
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10644//console
This message is automatically generated.
Restore ReaderBase.initAfterCompression() and
WALCellCodec.create(Configuration, CompressionContext)
Key: HBASE-11857
URL: https://issues.apache.org/jira/browse/HBASE-11857
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 11857-trunk-v1.txt, 11857-trunk-v2.txt,
11857-trunk-v3.txt
ReaderBase.initAfterCompression() and WALCellCodec.create(Configuration,
CompressionContext) were removed when adding Cell codec to WAL header.
Some downstream projects, such as Phoenix, depend on these methods.
This JIRA is to restore these methods.
Thanks to Jeffrey who found this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11689: Status: Open (was: Patch Available) Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 1.0.0, 2.0.0 Reporter: Jimmy Xiang Assignee: Andrey Stepachev Attachments: HBASE-11689-branch-1.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, HBASE-11689.patch, hbase-11689-revised.patch With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115646#comment-14115646 ] Mikhail Antonov commented on HBASE-11165: - [~stack] regarding multi-master..yeah, I'd make sure that it doesn't complicate the deployment (and to reduce scope of changes, that would be gradually in the direction of cold backup - warm - hot - active-active). Scaling so cluster can host 1M regions and beyond (50M regions?) Key: HBASE-11165 URL: https://issues.apache.org/jira/browse/HBASE-11165 Project: HBase Issue Type: Brainstorming Reporter: stack Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf This discussion issue comes out of Co-locate Meta And Master HBASE-10569 and comments on the doc posted there. A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end). More detail to follow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11760) Tighten up region state transition
[ https://issues.apache.org/jira/browse/HBASE-11760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115654#comment-14115654 ] stack commented on HBASE-11760: --- Skimming patch, it looks good. Let me give you a better review. What kind of testing have you done [~jxiang] Tighten up region state transition -- Key: HBASE-11760 URL: https://issues.apache.org/jira/browse/HBASE-11760 Project: HBase Issue Type: Improvement Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 2.0.0 Attachments: hbase-11760.patch, hbase-11760_2.patch When a regionserver reports to master a region transition, we should check the current region state to be exactly what we expect. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11760) Tighten up region state transition
[ https://issues.apache.org/jira/browse/HBASE-11760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115677#comment-14115677 ] Jimmy Xiang commented on HBASE-11760: - Thanks. I ran unit tests and ITBLL with DLR off. Tighten up region state transition -- Key: HBASE-11760 URL: https://issues.apache.org/jira/browse/HBASE-11760 Project: HBase Issue Type: Improvement Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 2.0.0 Attachments: hbase-11760.patch, hbase-11760_2.patch When a regionserver reports to master a region transition, we should check the current region state to be exactly what we expect. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11777) Find a way to use KV.setSequenceId() on Cells on the server-side read path
[
https://issues.apache.org/jira/browse/HBASE-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115694#comment-14115694
]
Hadoop QA commented on HBASE-11777:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665375/HBASE-11777_V3.patch
against trunk revision .
ATTACHMENT ID: 12665375
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:358)
at
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatTestBase.testWithMapReduce(TableSnapshotInputFormatTestBase.java:114)
at
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatTestBase.testWithMapReduceMultiRegion(TableSnapshotInputFormatTestBase.java:95)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10646//console
This message is automatically generated.
Find a way to use KV.setSequenceId() on Cells on the server-side read path
--
Key: HBASE-11777
URL: https://issues.apache.org/jira/browse/HBASE-11777
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 2.0.0
Attachments: CellWithSequenceNumber.java, HBASE-11777.patch,
HBASE-11777_V2.patch, HBASE-11777_V3.patch
Over in HBASE-11591 there was a need to set the sequenceId of the HFile to
the bulk loaded KVs. Since we are trying to use the concept of Cells in the
read path if we need to use setSequenceId(), then the Cell has to be
converted to KV and only KeyValue impl has the accessor setSequenceId().
[~anoop.hbase] suggested if we can use a Server side impl of Cell and have
these accessors in them.
This JIRA aims to solve this and see the related code changes that needs to
be carried out for this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115713#comment-14115713
]
Hudson commented on HBASE-11726:
FAILURE: Integrated in HBase-TRUNK #5447 (See
[https://builds.apache.org/job/HBase-TRUNK/5447/])
HBASE-11726 Master should fail-safe if starting with a pre 0.96 layout (Esteban
Guiterrez) (stack: rev 2cd45eb9a006c67f0aee76745cab85ea21f66e50)
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.7
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11862) Get rid of Writables in HTableDescriptor, HColumnDescriptor
Andrey Stepachev created HBASE-11862: Summary: Get rid of Writables in HTableDescriptor, HColumnDescriptor Key: HBASE-11862 URL: https://issues.apache.org/jira/browse/HBASE-11862 Project: HBase Issue Type: Improvement Reporter: Andrey Stepachev Priority: Minor Currently we have protobuf for encoding this structures. Existence of Writable is misleading and need to be removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11826) Split each tableOrRegionName admin methods into two targetted methods
[
https://issues.apache.org/jira/browse/HBASE-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115737#comment-14115737
]
Hadoop QA commented on HBASE-11826:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12665382/HBASE_11826_v2.patch
against trunk revision .
ATTACHMENT ID: 12665382
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10647//console
This message is automatically generated.
Split each tableOrRegionName admin methods into two targetted methods
-
Key: HBASE-11826
URL: https://issues.apache.org/jira/browse/HBASE-11826
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
Attachments: HBASE_11826.patch, HBASE_11826_v2.patch,
HBASE_11826_v2.patch
Purpose of this is two implement [~enis]'s suggestion to strongly type the
methods that take tableOrRegionName as an argument.
For instance:
{code}
void compact(final String tableNameOrRegionName)
void compact(final byte[] tableNameOrRegionName)
{code}
becomes
{code}
@Deprecated
void compact(final String tableNameOrRegionName)
@Deprecated
void compact(final byte[] tableNameOrRegionName)
void compact(TableName table)
void compactRegion(final byte[] regionName)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11861) Native MOB Compaction mechanisms.
[ https://issues.apache.org/jira/browse/HBASE-11861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11861: --- Description: Currently, the first cut of mob will have external processes to age off old mob data (the ttl cleaner), and to compact away deleted or over written data (the sweep tool). From an operational point of view, having two external tools, especially one that relies on MapReduce is undesirable. In this issue we'll tackle integrating these into hbase without requiring external processes. was: Currenlty, the first cut of mob will have external processes to age off old mob data (the ttl cleaner), and to compact away deleted or over written data (the sweep tool). From an operational point of view, having two external tools, especially one that relies on MapReduce is undesirable. In this issue we'll tackle integrating these into hbase without requiring external processes. Native MOB Compaction mechanisms. - Key: HBASE-11861 URL: https://issues.apache.org/jira/browse/HBASE-11861 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Currently, the first cut of mob will have external processes to age off old mob data (the ttl cleaner), and to compact away deleted or over written data (the sweep tool). From an operational point of view, having two external tools, especially one that relies on MapReduce is undesirable. In this issue we'll tackle integrating these into hbase without requiring external processes. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11837) Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no local data
[
https://issues.apache.org/jira/browse/HBASE-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115756#comment-14115756
]
Hudson commented on HBASE-11837:
SUCCESS: Integrated in HBase-0.98 #485 (See
[https://builds.apache.org/job/HBase-0.98/485/])
HBASE-11837: Scanner from RegionCoprocessorEnvironment.getTable(TableName)
returns no local data (jeffreyz: rev fd35f67040dce49bebb6f01db3b402eac136b90d)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/client/CoprocessorHConnection.java
Scanner from RegionCoprocessorEnvironment.getTable(TableName) returns no
local data
---
Key: HBASE-11837
URL: https://issues.apache.org/jira/browse/HBASE-11837
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.98.4
Reporter: James Taylor
Assignee: Jeffrey Zhong
Fix For: 0.98.7
Attachments: HBASE-11837.patch, PHOENIX-1208-torepro.patch
Might be fixed as a byproduct of the fix for HBASE-11766.
Here's what I'm seeing:
- From an endpoint coprocessor on SYSTEM.CATALOG table
- Rows exist in the table, but are all in a single region
- Scan is a new, empty scan created like this: new Scan()
- The following scanner *does* return data:
{code}
RegionScanner scanner = region.getScanner(scan);
{code}
- The following scanner *does not* return any data:
{code}
HTableInterface hTable = env.getTable(region.getTableDesc().getTableName());
{code}
- The following scanner *does* return data:
{code}
HTablePool pool = new HTablePool (env.getConfiguration(),1);
HTableInterface hTable = pool.getTable(SYSTEM.CATALOG);
{code}
[~jeffreyz], [~apurtell]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11726) Master should fail-safe if starting with a pre 0.96 layout
[
https://issues.apache.org/jira/browse/HBASE-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115755#comment-14115755
]
Hudson commented on HBASE-11726:
SUCCESS: Integrated in HBase-0.98 #485 (See
[https://builds.apache.org/job/HBase-0.98/485/])
HBASE-11726 Master should fail-safe if starting with a pre 0.96 layout (Esteban
Guiterrez) (stack: rev 6a775e087f7579f93b65cc3828e369dd04896cf9)
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
Master should fail-safe if starting with a pre 0.96 layout
--
Key: HBASE-11726
URL: https://issues.apache.org/jira/browse/HBASE-11726
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.96.2, 0.99.0, 0.98.5, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.7
Attachments: HBASE-11726.v0.diff, HBASE-11726.v1.diff,
HBASE-11726.v2.diff
We recently saw this: If user inadvertently starts the HBase Master after
deploying new HBase binaries (any version that supports namespaces), the
HMaster will start the migration to PBs the the {{hbase.version}} file per
HBASE-5453 and that will write a new version file PB-serialized but with the
old version number. Further restarts of the master will fail because the
hbase version file has been migrated to PBs and there will be version
mismatch. The right approach should be to fail safe the master if we find an
old {{hbase.version}} file in order to force user to run upgrade tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
