Re: Planning for HBase 1.2
It's a large new feature. We might not technically break anything, but we're introducing a lot of extra risk.As a case in point... At Salesforce we think the timing would work out to make 1.2 our next standard version. If MOBs were merged we'd likely stick with 1.1 instead. I guess it all depends how long we're planning to maintain branches. If 1.2 indicates the soon death of 1.1 we need to be more careful in 1.2. If we're planning to keep 1.1 (and 1.0) around as long as 0.98, we can be more aggressive. It's a fluffy discussion :) -- Lars From: Sean Busbey To: dev ; lars hofhansl Sent: Friday, May 22, 2015 9:45 PM Subject: Re: Planning for HBase 1.2 So long as we can do it without breaking compatibility, what feels wrong about it? If our goal is to get to semver, we have to progress to descriptive numbers at some point. On Fri, May 22, 2015 at 11:39 PM, lars hofhansl wrote: Agreed. I'd add that shipping a major feature like MOB with a minor branch "feels" wrong. From: Andrew Purtell To: "dev@hbase.apache.org" Sent: Friday, May 22, 2015 12:40 PM Subject: Re: Planning for HBase 1.2 Another point of clarification, sorry, I hit the send button too early it seems: I don't believe MOB is fully integrated yet, for example the feature is an extension to store that lacks support for encryption (this would technically be a feature regression); and HBCK. I have not been following MOB too closely so could be mistaken. These issues do not preclude a merge of MOB into trunk, but do preclude a merge back of MOB from trunk to branch-1. I would veto the latter until such shortcomings in the implementation that could be described as regressions are addressed. I would also like to see a performance analysis of a range of workloads before and after in as much detail as can be mustered, and would be happy to volunteer to help out with that. On Fri, May 22, 2015 at 12:32 PM, Andrew Purtell wrote: > I was also thinking about RMing for 1.2 as we try and bring something post > 1.0 into production at my employer. > > Related, of the list of features proposed I would strongly prefer MOB not > be included. > > > > On Fri, May 22, 2015 at 12:19 PM, Sean Busbey wrote: > >> Hi folks! >> >> I'd like to volunteer to RM HBase 1.2 and aim for RCs starting in July. >> >> Here's an initial list of things I want to get out: >> >> * MOB >> >> * native crc >> >> * incremental improvements for procedure v2 >> >> * adding Java 8 as supported >> >> >> Anything else folks want to see called out? >> >> -- >> Sean >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Sean
Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
I was responding to this comment from Jon's email: > Another suggestion was a tool to check that mob references had > corresponding mob data. We currently include a mr-based sweeper job > that could be used to perform this verification. We can add this tool and > testing for the tool. So for those of us not intimately familiar with the MOB work, is there a tool that checks for MOB integrity, or a tool that can be adapted for that purpose, and does it require MR or not? More generally: Can MOB integrity checks be added or folded into HBCK? I think you can see what my concerns are but if they are unclear please let me know and I will clarify them further. On Sat, May 23, 2015 at 11:15 AM, Matteo Bertozzi wrote: > as far as I know MOB does not depend anymore on MR > the old MR sweeper tool is still around, and you can use it to compact > manually > but it is not called by the normal RS compaction code. > > also, the MOB code is more or less isolated. > if your family is not using MOB you still have your old code path. > so, I'd say that if we don't break compatibility and > the few changes in the core-path, to do the if mobIsEnabled, do not impact > the perf of the traditional path > we can probably get the feature in 1.2 as "experimental". > brave users can experiment with it, report bugs and suggestions > and then we will mark it as stable in 1.3, 1.4 or whenever is ready. > > > Matteo > > > On Sat, May 23, 2015 at 9:47 AM, Andrew Purtell > wrote: > > > Maybe we can remove the dependency on a MR runtime for MOB maintenance by > > reimplementing those parallel tasks using Procedure V2? We wouldn't be > > looking at MOB for 1.2 but maybe 1.3? I'm also not sure the community as > a > > whole has the necessary bandwidth for perf and stability testing of MOB > in > > the 1.2 timeframe, but 1.3 would be more likely. > > > > > > On Sat, May 23, 2015 at 9:40 AM, Andrew Purtell > > wrote: > > > > > Regarding performance testing: Whatever has been done on the MOB branch > > > will be interesting data points, and, potentially encouraging, but > > porting > > > to branch-1 will produce a new code base. Earlier results on other code > > > will not be applicable. We have to start over. Like I said elsewhere, > I'm > > > happy to help with (re)characterizing the perf impact and improvements > > > produced by the changes. > > > > > > What coverage do we have for verifying the integrity of MOB references? > > > Will the sweep tool detect, alert on, and optionally repair dangling > > > references? (I could answer this for myself by looking at MOB branch, > but > > > hopefully someone here has an answer at the ready.) I assume we > calculate > > > and store checksums for MOB data itself so we know if values are > corrupt. > > > Does the sweep tool detect MOB value corruption? Can it be repaired? Do > > we > > > have a good ops story for why HBCK is no longer sufficient on its own, > > > there's a separate tool with a whole new set of options - and a > > requirement > > > for a MR runtime! - for checking MOB data? That last one is a > rhetorical > > > question (smile), the ops story is... unsatisfying. It's like we've > > taken a > > > self sufficient HBase and bolted in parts of Hive, so now we need MR. > > > > > > > > > On Fri, May 22, 2015 at 1:45 PM, Jonathan Hsieh > > wrote: > > > > > >> In another thread andrew purtell brought up some concerns about the > mob > > >> feature: > > >> > > >> On Fri, May 22, 2015 at 12:40 PM, Andrew Purtell > > > >> wrote: > > >> > > >> > Another point of clarification, sorry, I hit the send button too > early > > >> it > > >> > seems: I don't believe MOB is fully integrated yet, for example the > > >> > feature > > >> > is an extension to store that lacks support for encryption (this > would > > >> > technically be a feature regression); and HBCK. I have not been > > >> following > > >> > MOB too closely so could be mistaken. These issues do not preclude a > > >> merge > > >> > of MOB into trunk, but do preclude a merge back of MOB from trunk to > > >> > branch-1. I would veto the latter until such shortcomings in the > > >> > implementation that could be described as regressions are > addressed. I > > >> > would also like to see a performance analysis of a range of > workloads > > >> > before and after in as much detail as can be mustered, and would be > > >> happy > > >> > to volunteer to help out with that. > > >> > > > >> > > >> Here's info on the points brought up: > > >> > > >> Encryption support shortcoming is being addrsessed here: > > >> https://issues.apache.org/jira/browse/HBASE-13693 (closed) > > >> https://issues.apache.org/jira/browse/HBASE-13720 (in review) > > >> > > >> Hbck has been actually run against the integration test rigs while the > > >> feature has been enabled but currently has no explicit unit test or > > simple > > >> to run integration test. It currently doesn't report anything special > > >> about the mob storage area. We can add unit tests tha
[jira] [Created] (HBASE-13757) TestMultiParallel (and others) failing on 0.98 since HBASE-13712
Andrew Purtell created HBASE-13757: -- Summary: TestMultiParallel (and others) failing on 0.98 since HBASE-13712 Key: HBASE-13757 URL: https://issues.apache.org/jira/browse/HBASE-13757 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.98.13 I recently backported a bunch of changes to the Canary to 0.98, and as a standalone daemon it tested out ok. On HBASE-13712 the reviewer and I mistook precommit failures for an ASF Jenkins botch but several test failures were legitimate. Fixing them here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
as far as I know MOB does not depend anymore on MR the old MR sweeper tool is still around, and you can use it to compact manually but it is not called by the normal RS compaction code. also, the MOB code is more or less isolated. if your family is not using MOB you still have your old code path. so, I'd say that if we don't break compatibility and the few changes in the core-path, to do the if mobIsEnabled, do not impact the perf of the traditional path we can probably get the feature in 1.2 as "experimental". brave users can experiment with it, report bugs and suggestions and then we will mark it as stable in 1.3, 1.4 or whenever is ready. Matteo On Sat, May 23, 2015 at 9:47 AM, Andrew Purtell wrote: > Maybe we can remove the dependency on a MR runtime for MOB maintenance by > reimplementing those parallel tasks using Procedure V2? We wouldn't be > looking at MOB for 1.2 but maybe 1.3? I'm also not sure the community as a > whole has the necessary bandwidth for perf and stability testing of MOB in > the 1.2 timeframe, but 1.3 would be more likely. > > > On Sat, May 23, 2015 at 9:40 AM, Andrew Purtell > wrote: > > > Regarding performance testing: Whatever has been done on the MOB branch > > will be interesting data points, and, potentially encouraging, but > porting > > to branch-1 will produce a new code base. Earlier results on other code > > will not be applicable. We have to start over. Like I said elsewhere, I'm > > happy to help with (re)characterizing the perf impact and improvements > > produced by the changes. > > > > What coverage do we have for verifying the integrity of MOB references? > > Will the sweep tool detect, alert on, and optionally repair dangling > > references? (I could answer this for myself by looking at MOB branch, but > > hopefully someone here has an answer at the ready.) I assume we calculate > > and store checksums for MOB data itself so we know if values are corrupt. > > Does the sweep tool detect MOB value corruption? Can it be repaired? Do > we > > have a good ops story for why HBCK is no longer sufficient on its own, > > there's a separate tool with a whole new set of options - and a > requirement > > for a MR runtime! - for checking MOB data? That last one is a rhetorical > > question (smile), the ops story is... unsatisfying. It's like we've > taken a > > self sufficient HBase and bolted in parts of Hive, so now we need MR. > > > > > > On Fri, May 22, 2015 at 1:45 PM, Jonathan Hsieh > wrote: > > > >> In another thread andrew purtell brought up some concerns about the mob > >> feature: > >> > >> On Fri, May 22, 2015 at 12:40 PM, Andrew Purtell > >> wrote: > >> > >> > Another point of clarification, sorry, I hit the send button too early > >> it > >> > seems: I don't believe MOB is fully integrated yet, for example the > >> > feature > >> > is an extension to store that lacks support for encryption (this would > >> > technically be a feature regression); and HBCK. I have not been > >> following > >> > MOB too closely so could be mistaken. These issues do not preclude a > >> merge > >> > of MOB into trunk, but do preclude a merge back of MOB from trunk to > >> > branch-1. I would veto the latter until such shortcomings in the > >> > implementation that could be described as regressions are addressed. I > >> > would also like to see a performance analysis of a range of workloads > >> > before and after in as much detail as can be mustered, and would be > >> happy > >> > to volunteer to help out with that. > >> > > >> > >> Here's info on the points brought up: > >> > >> Encryption support shortcoming is being addrsessed here: > >> https://issues.apache.org/jira/browse/HBASE-13693 (closed) > >> https://issues.apache.org/jira/browse/HBASE-13720 (in review) > >> > >> Hbck has been actually run against the integration test rigs while the > >> feature has been enabled but currently has no explicit unit test or > simple > >> to run integration test. It currently doesn't report anything special > >> about the mob storage area. We can add unit tests that cover hbck when > the > >> mob path is exercised. > >> > >> Another suggestion was a tool to check that mob references had > >> corresponding mob data. We currently include a mr-based sweeper job > that > >> could be used to perform this verification. We can add this tool and > >> testing for the tool. > >> > >> I've done some performance testing and Jingcheng and his colleagues have > >> done significant amounts of performance testing. We currently have a > blog > >> post in progress that will share the results of this performance > testing. > >> > >> Jon. > >> > >> > >> > >> > >> > >> On Wed, May 20, 2015 at 7:38 PM, Ted Yu wrote: > >> > >> > This is a useful feature, Jon. > >> > > >> > I went over the mega-patch and left some comments on review board. > >> > > >> > I noticed that hbck was not included in the patch. Neither did I find > a > >> > sub-task of HBASE-11339 that covers hbck. > >> > > >> > Do you or Jing
[jira] [Created] (HBASE-13756) Region server is getting aborted after with NoSuchMethodError when RS reporting to master
Rajeshbabu Chintaguntla created HBASE-13756:
---
Summary: Region server is getting aborted after with
NoSuchMethodError when RS reporting to master
Key: HBASE-13756
URL: https://issues.apache.org/jira/browse/HBASE-13756
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Priority: Critical
I have observed below exception when running Phoenix integration tests with
HBase-1.1.0. I think same can happen in real cluster when RS reporting to
master.
{noformat}
ABORTING region server 100.73.163.39,53415,1432394107922: Unhandled:
org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ServerLoad$Builder.setNumberOfRequests(J)Lorg/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos$ServerLoad$Builder;
Cause:
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ServerLoad$Builder.setNumberOfRequests(J)Lorg/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos$ServerLoad$Builder;
at
org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:1165)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1127)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:944)
at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:156)
at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:108)
at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:140)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:306)
at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:138)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Maybe we can remove the dependency on a MR runtime for MOB maintenance by reimplementing those parallel tasks using Procedure V2? We wouldn't be looking at MOB for 1.2 but maybe 1.3? I'm also not sure the community as a whole has the necessary bandwidth for perf and stability testing of MOB in the 1.2 timeframe, but 1.3 would be more likely. On Sat, May 23, 2015 at 9:40 AM, Andrew Purtell wrote: > Regarding performance testing: Whatever has been done on the MOB branch > will be interesting data points, and, potentially encouraging, but porting > to branch-1 will produce a new code base. Earlier results on other code > will not be applicable. We have to start over. Like I said elsewhere, I'm > happy to help with (re)characterizing the perf impact and improvements > produced by the changes. > > What coverage do we have for verifying the integrity of MOB references? > Will the sweep tool detect, alert on, and optionally repair dangling > references? (I could answer this for myself by looking at MOB branch, but > hopefully someone here has an answer at the ready.) I assume we calculate > and store checksums for MOB data itself so we know if values are corrupt. > Does the sweep tool detect MOB value corruption? Can it be repaired? Do we > have a good ops story for why HBCK is no longer sufficient on its own, > there's a separate tool with a whole new set of options - and a requirement > for a MR runtime! - for checking MOB data? That last one is a rhetorical > question (smile), the ops story is... unsatisfying. It's like we've taken a > self sufficient HBase and bolted in parts of Hive, so now we need MR. > > > On Fri, May 22, 2015 at 1:45 PM, Jonathan Hsieh wrote: > >> In another thread andrew purtell brought up some concerns about the mob >> feature: >> >> On Fri, May 22, 2015 at 12:40 PM, Andrew Purtell >> wrote: >> >> > Another point of clarification, sorry, I hit the send button too early >> it >> > seems: I don't believe MOB is fully integrated yet, for example the >> > feature >> > is an extension to store that lacks support for encryption (this would >> > technically be a feature regression); and HBCK. I have not been >> following >> > MOB too closely so could be mistaken. These issues do not preclude a >> merge >> > of MOB into trunk, but do preclude a merge back of MOB from trunk to >> > branch-1. I would veto the latter until such shortcomings in the >> > implementation that could be described as regressions are addressed. I >> > would also like to see a performance analysis of a range of workloads >> > before and after in as much detail as can be mustered, and would be >> happy >> > to volunteer to help out with that. >> > >> >> Here's info on the points brought up: >> >> Encryption support shortcoming is being addrsessed here: >> https://issues.apache.org/jira/browse/HBASE-13693 (closed) >> https://issues.apache.org/jira/browse/HBASE-13720 (in review) >> >> Hbck has been actually run against the integration test rigs while the >> feature has been enabled but currently has no explicit unit test or simple >> to run integration test. It currently doesn't report anything special >> about the mob storage area. We can add unit tests that cover hbck when the >> mob path is exercised. >> >> Another suggestion was a tool to check that mob references had >> corresponding mob data. We currently include a mr-based sweeper job that >> could be used to perform this verification. We can add this tool and >> testing for the tool. >> >> I've done some performance testing and Jingcheng and his colleagues have >> done significant amounts of performance testing. We currently have a blog >> post in progress that will share the results of this performance testing. >> >> Jon. >> >> >> >> >> >> On Wed, May 20, 2015 at 7:38 PM, Ted Yu wrote: >> >> > This is a useful feature, Jon. >> > >> > I went over the mega-patch and left some comments on review board. >> > >> > I noticed that hbck was not included in the patch. Neither did I find a >> > sub-task of HBASE-11339 that covers hbck. >> > >> > Do you or Jingcheng plan to add MOB-aware capability for hbck ? >> > >> > Cheers >> > >> > On Wed, May 20, 2015 at 9:21 AM, Jonathan Hsieh >> wrote: >> > >> > > Hi folks, >> > > >> > > The Medium Object (MOB) Storage feature (HBASE-11339[1]) is modified >> I/O >> > > and compaction path that allows individual moderately sized values >> > > (10k-10MB) to be stored so that write amplification is reduced when >> > > compared to the normal I/O path. At a high level, it provides >> alternate >> > > flush and compaction mechanisms that segregates large cells into a >> > separate >> > > area where they are not subject to potentially frequent compaction and >> > > splits that can be encountered in the normal I/O path. A more detailed >> > > design doc can be found on the hbase-11339 jira. >> > > >> > > Jingcheng Du has been working on the mob feature for a while and >> Anoop, >> > Ram >> > > and I have been shepherding him throug
Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Regarding performance testing: Whatever has been done on the MOB branch will be interesting data points, and, potentially encouraging, but porting to branch-1 will produce a new code base. Earlier results on other code will not be applicable. We have to start over. Like I said elsewhere, I'm happy to help with (re)characterizing the perf impact and improvements produced by the changes. What coverage do we have for verifying the integrity of MOB references? Will the sweep tool detect, alert on, and optionally repair dangling references? (I could answer this for myself by looking at MOB branch, but hopefully someone here has an answer at the ready.) I assume we calculate and store checksums for MOB data itself so we know if values are corrupt. Does the sweep tool detect MOB value corruption? Can it be repaired? Do we have a good ops story for why HBCK is no longer sufficient on its own, there's a separate tool with a whole new set of options - and a requirement for a MR runtime! - for checking MOB data? That last one is a rhetorical question (smile), the ops story is... unsatisfying. It's like we've taken a self sufficient HBase and bolted in parts of Hive, so now we need MR. On Fri, May 22, 2015 at 1:45 PM, Jonathan Hsieh wrote: > In another thread andrew purtell brought up some concerns about the mob > feature: > > On Fri, May 22, 2015 at 12:40 PM, Andrew Purtell > wrote: > > > Another point of clarification, sorry, I hit the send button too early it > > seems: I don't believe MOB is fully integrated yet, for example the > > feature > > is an extension to store that lacks support for encryption (this would > > technically be a feature regression); and HBCK. I have not been following > > MOB too closely so could be mistaken. These issues do not preclude a > merge > > of MOB into trunk, but do preclude a merge back of MOB from trunk to > > branch-1. I would veto the latter until such shortcomings in the > > implementation that could be described as regressions are addressed. I > > would also like to see a performance analysis of a range of workloads > > before and after in as much detail as can be mustered, and would be happy > > to volunteer to help out with that. > > > > Here's info on the points brought up: > > Encryption support shortcoming is being addrsessed here: > https://issues.apache.org/jira/browse/HBASE-13693 (closed) > https://issues.apache.org/jira/browse/HBASE-13720 (in review) > > Hbck has been actually run against the integration test rigs while the > feature has been enabled but currently has no explicit unit test or simple > to run integration test. It currently doesn't report anything special > about the mob storage area. We can add unit tests that cover hbck when the > mob path is exercised. > > Another suggestion was a tool to check that mob references had > corresponding mob data. We currently include a mr-based sweeper job that > could be used to perform this verification. We can add this tool and > testing for the tool. > > I've done some performance testing and Jingcheng and his colleagues have > done significant amounts of performance testing. We currently have a blog > post in progress that will share the results of this performance testing. > > Jon. > > > > > > On Wed, May 20, 2015 at 7:38 PM, Ted Yu wrote: > > > This is a useful feature, Jon. > > > > I went over the mega-patch and left some comments on review board. > > > > I noticed that hbck was not included in the patch. Neither did I find a > > sub-task of HBASE-11339 that covers hbck. > > > > Do you or Jingcheng plan to add MOB-aware capability for hbck ? > > > > Cheers > > > > On Wed, May 20, 2015 at 9:21 AM, Jonathan Hsieh > wrote: > > > > > Hi folks, > > > > > > The Medium Object (MOB) Storage feature (HBASE-11339[1]) is modified > I/O > > > and compaction path that allows individual moderately sized values > > > (10k-10MB) to be stored so that write amplification is reduced when > > > compared to the normal I/O path. At a high level, it provides > alternate > > > flush and compaction mechanisms that segregates large cells into a > > separate > > > area where they are not subject to potentially frequent compaction and > > > splits that can be encountered in the normal I/O path. A more detailed > > > design doc can be found on the hbase-11339 jira. > > > > > > Jingcheng Du has been working on the mob feature for a while and Anoop, > > Ram > > > and I have been shepherding him through the design revisions and > > > implementation of the feature in the hbase-11339 branch.[2] > > > > > > The branch we are proposing to merge into master is compatible with > > HBase's > > > core functionality including snapshots, replication, shell support, > > behaves > > > well with table alters, bulk loads and does not require external MR > > > processes. It has been documented, and subject to many integration test > > > runs (ITBLL, ITAcidGuarantees, ITIngest) including fault injection. > > > Performance testing of the featu
[jira] [Created] (HBASE-13755) Provide single super user check implementation
Anoop Sam John created HBASE-13755: -- Summary: Provide single super user check implementation Key: HBASE-13755 URL: https://issues.apache.org/jira/browse/HBASE-13755 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Followup for HBASE-13375. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Planning for HBase 1.2
>Lack of support for HBase native at rest encryption is something that could be documented. FYI.. One defect is closed.. But we missed handling some cases (MOB sweep tool) and for that also there is a patch available. It will get committed in a day.. -Anoop- On Sat, May 23, 2015 at 10:15 AM, Sean Busbey wrote: > So long as we can do it without breaking compatibility, what feels wrong > about it? If our goal is to get to semver, we have to progress to > descriptive numbers at some point. > > On Fri, May 22, 2015 at 11:39 PM, lars hofhansl wrote: > > > Agreed. I'd add that shipping a major feature like MOB with a minor > branch > > "feels" wrong. > > From: Andrew Purtell > > To: "dev@hbase.apache.org" > > Sent: Friday, May 22, 2015 12:40 PM > > Subject: Re: Planning for HBase 1.2 > > > > Another point of clarification, sorry, I hit the send button too early it > > seems: I don't believe MOB is fully integrated yet, for example the > feature > > is an extension to store that lacks support for encryption (this would > > technically be a feature regression); and HBCK. I have not been following > > MOB too closely so could be mistaken. These issues do not preclude a > merge > > of MOB into trunk, but do preclude a merge back of MOB from trunk to > > branch-1. I would veto the latter until such shortcomings in the > > implementation that could be described as regressions are addressed. I > > would also like to see a performance analysis of a range of workloads > > before and after in as much detail as can be mustered, and would be happy > > to volunteer to help out with that. > > > > > > On Fri, May 22, 2015 at 12:32 PM, Andrew Purtell > > wrote: > > > > > I was also thinking about RMing for 1.2 as we try and bring something > > post > > > 1.0 into production at my employer. > > > > > > Related, of the list of features proposed I would strongly prefer MOB > not > > > be included. > > > > > > > > > > > > On Fri, May 22, 2015 at 12:19 PM, Sean Busbey > > wrote: > > > > > >> Hi folks! > > >> > > >> I'd like to volunteer to RM HBase 1.2 and aim for RCs starting in > July. > > >> > > >> Here's an initial list of things I want to get out: > > >> > > >> * MOB > > >> > > >> * native crc > > >> > > >> * incremental improvements for procedure v2 > > >> > > >> * adding Java 8 as supported > > >> > > >> > > >> Anything else folks want to see called out? > > >> > > >> -- > > >> Sean > > >> > > > > > > > > > > > > -- > > > Best regards, > > > > > >- Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) > > > > > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > > > > > > > > > > > -- > Sean >
[jira] [Created] (HBASE-13754) Allow non KeyValue Cell types also to oswrite
Anoop Sam John created HBASE-13754: -- Summary: Allow non KeyValue Cell types also to oswrite Key: HBASE-13754 URL: https://issues.apache.org/jira/browse/HBASE-13754 Project: HBase Issue Type: Sub-task Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0 While making the cellblock for returning data to client, we have to write the cell data into an OutputStream. KeyValue has a static oswrite() method with which it can write data in one go. (KeyValue components are in a single byte[]). For other cell implementation, we will call getXXXLength() and getXXXArray() and write each component one after the other. This is not efficient as the KeyValue way. In fact other cell impls also may have one contigous byte[] backing for keys atleast. (See ClonedSeekerState) We can optimize for such Cells also. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
