[RESULT] [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
This merge vote passes with 4 binding +1's and 3 non-binding +1's. Thanks everyone. I'll go ahead with the merge into master and see how easily it comes back to branch-2. On 2021/12/16 21:30:09 Josh Elser wrote: Hi! I'm extremely pleased to send this official vote to merge the feature branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). This feature branch introduces the pluggable StoreFileTracker interface. The StoreFileTracker allows the StoreFileEngine to be decoupled from where the HFiles can be found. The DEFAULT implementation of this StoreFileTracker (e.g. files in a family's directory) is still the default implementation. This merge would introduce a FILE implementation which uses a flat-file in each column family to track the files which make up this Store. This feature is notable for HBase as it invalidate the need for HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is used for HBase data. We had a DISCUSS thread [2] in which the overall sentiment was positive to merge. Covering some high-level details/FAQ on this work: * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies using this feature. * YCSB (load) indicates a slight performance improvement when using S3 as the storage system for HBase as compared to using HBOSS [3] * A new section was added to the HBase book which covers the feature and how to use it. * There is some follow-on work expected, tracked in HBASE-26584 [4], which includes things like making user consumption easier and additional metrics to measure effectiveness of the feature. As is customary, this vote will be open for at least 3 days (2021/12/19 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code modification (though I do not expect to need the lazy-consensus caveat). Please vote: +1: Merge the changes from HBASE-26067 to master and branch-2 -1: Do not merge these changes because ... Big thank you to all of the super hard work that Duo, Wellington, and Szabolcs have put into this feature. - Josh [1] https://issues.apache.org/jira/browse/HBASE-26067 [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 [3] https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 [4] https://issues.apache.org/jira/browse/HBASE-26584
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
Duo -- yes, you are correct, but I wanted to give the opportunity for others who didn't work on it (and thus, have an interest in seeing it merged) to also vote :). I plan to merge this into master given the current response from the other devs. Andrew -- I feel that very well. Thanks for taking the time to drop an email! On Tue, Dec 21, 2021 at 5:09 PM Andrew Purtell wrote: > > +1 (binding) > > Really happy to see this. Apologies that I have so limited bandwidth these > days. > > > On Mon, Dec 20, 2021 at 10:54 AM Josh Elser wrote: > > > Thanks to all who took the time to vote already! > > > > So far, we have 5 +1's (2 binding, 3 non-binding) which is sufficient > > for a lazy-consensus RESULT. However, I'd love to see a vote from > > someone who didn't contribute code to this feature. I know running an > > in-depth technical analysis is a major undertaking. I'd ask for a third > > binding vote even if someone just does a high-level review of the work > > on the feature branch. > > > > At the same time, I also know many folks will be relaxing with their > > families. I'll leave this open for 2 more days in the hopes that some > > other folks will still have time to weigh in. > > > > On 2021/12/16 21:30:09 Josh Elser wrote: > > > Hi! > > > > > > I'm extremely pleased to send this official vote to merge the feature > > > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > > > not 2.5.x). This feature branch introduces the pluggable > > > StoreFileTracker interface. > > > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > > > where the HFiles can be found. The DEFAULT implementation of this > > > StoreFileTracker (e.g. files in a family's directory) is still the > > > default implementation. This merge would introduce a FILE implementation > > > which uses a flat-file in each column family to track the files which > > > make up this Store. This feature is notable for HBase as it invalidate > > > the need for HBOSS (a distributed locking layer in hbase-filesystem) > > > when Amazon S3 is used for HBase data. > > > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > > > to merge. > > > > > > Covering some high-level details/FAQ on this work: > > > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies > > > using this feature. > > > * YCSB (load) indicates a slight performance improvement when using S3 > > > as the storage system for HBase as compared to using HBOSS [3] > > > * A new section was added to the HBase book which covers the feature and > > > how to use it. > > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > > > which includes things like making user consumption easier and additional > > > metrics to measure effectiveness of the feature. > > > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > > > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > > > modification (though I do not expect to need the lazy-consensus caveat). > > > Please vote: > > > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > > -1: Do not merge these changes because ... > > > > > > Big thank you to all of the super hard work that Duo, Wellington, and > > > Szabolcs have put into this feature. > > > > > > - Josh > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > > [3] > > > > > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
+1 (binding) Really happy to see this. Apologies that I have so limited bandwidth these days. On Mon, Dec 20, 2021 at 10:54 AM Josh Elser wrote: > Thanks to all who took the time to vote already! > > So far, we have 5 +1's (2 binding, 3 non-binding) which is sufficient > for a lazy-consensus RESULT. However, I'd love to see a vote from > someone who didn't contribute code to this feature. I know running an > in-depth technical analysis is a major undertaking. I'd ask for a third > binding vote even if someone just does a high-level review of the work > on the feature branch. > > At the same time, I also know many folks will be relaxing with their > families. I'll leave this open for 2 more days in the hopes that some > other folks will still have time to weigh in. > > On 2021/12/16 21:30:09 Josh Elser wrote: > > Hi! > > > > I'm extremely pleased to send this official vote to merge the feature > > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > > not 2.5.x). This feature branch introduces the pluggable > > StoreFileTracker interface. > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > > where the HFiles can be found. The DEFAULT implementation of this > > StoreFileTracker (e.g. files in a family's directory) is still the > > default implementation. This merge would introduce a FILE implementation > > which uses a flat-file in each column family to track the files which > > make up this Store. This feature is notable for HBase as it invalidate > > the need for HBOSS (a distributed locking layer in hbase-filesystem) > > when Amazon S3 is used for HBase data. > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > > to merge. > > > > Covering some high-level details/FAQ on this work: > > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies > > using this feature. > > * YCSB (load) indicates a slight performance improvement when using S3 > > as the storage system for HBase as compared to using HBOSS [3] > > * A new section was added to the HBase book which covers the feature and > > how to use it. > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > > which includes things like making user consumption easier and additional > > metrics to measure effectiveness of the feature. > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > > modification (though I do not expect to need the lazy-consensus caveat). > > Please vote: > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > -1: Do not merge these changes because ... > > > > Big thank you to all of the super hard work that Duo, Wellington, and > > Szabolcs have put into this feature. > > > > - Josh > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > [3] > > > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
IIRC for merging a feature branch back, 3 committers' +1 is enough? Peter Somogyi 于2021年12月21日周二 04:07写道： > +1 > > On Mon, Dec 20, 2021 at 7:54 PM Josh Elser wrote: > > > Thanks to all who took the time to vote already! > > > > So far, we have 5 +1's (2 binding, 3 non-binding) which is sufficient > > for a lazy-consensus RESULT. However, I'd love to see a vote from > > someone who didn't contribute code to this feature. I know running an > > in-depth technical analysis is a major undertaking. I'd ask for a third > > binding vote even if someone just does a high-level review of the work > > on the feature branch. > > > > At the same time, I also know many folks will be relaxing with their > > families. I'll leave this open for 2 more days in the hopes that some > > other folks will still have time to weigh in. > > > > On 2021/12/16 21:30:09 Josh Elser wrote: > > > Hi! > > > > > > I'm extremely pleased to send this official vote to merge the feature > > > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > > > not 2.5.x). This feature branch introduces the pluggable > > > StoreFileTracker interface. > > > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > > > where the HFiles can be found. The DEFAULT implementation of this > > > StoreFileTracker (e.g. files in a family's directory) is still the > > > default implementation. This merge would introduce a FILE > implementation > > > which uses a flat-file in each column family to track the files which > > > make up this Store. This feature is notable for HBase as it invalidate > > > the need for HBOSS (a distributed locking layer in hbase-filesystem) > > > when Amazon S3 is used for HBase data. > > > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > > > to merge. > > > > > > Covering some high-level details/FAQ on this work: > > > * Wellington and Szabolcs have successfully run ITBLL with Chaos > Monkies > > > using this feature. > > > * YCSB (load) indicates a slight performance improvement when using S3 > > > as the storage system for HBase as compared to using HBOSS [3] > > > * A new section was added to the HBase book which covers the feature > and > > > how to use it. > > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > > > which includes things like making user consumption easier and > additional > > > metrics to measure effectiveness of the feature. > > > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > > > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > > > modification (though I do not expect to need the lazy-consensus > caveat). > > > Please vote: > > > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > > -1: Do not merge these changes because ... > > > > > > Big thank you to all of the super hard work that Duo, Wellington, and > > > Szabolcs have put into this feature. > > > > > > - Josh > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > > [3] > > > > > > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > > > > > >
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
+1 On Mon, Dec 20, 2021 at 7:54 PM Josh Elser wrote: > Thanks to all who took the time to vote already! > > So far, we have 5 +1's (2 binding, 3 non-binding) which is sufficient > for a lazy-consensus RESULT. However, I'd love to see a vote from > someone who didn't contribute code to this feature. I know running an > in-depth technical analysis is a major undertaking. I'd ask for a third > binding vote even if someone just does a high-level review of the work > on the feature branch. > > At the same time, I also know many folks will be relaxing with their > families. I'll leave this open for 2 more days in the hopes that some > other folks will still have time to weigh in. > > On 2021/12/16 21:30:09 Josh Elser wrote: > > Hi! > > > > I'm extremely pleased to send this official vote to merge the feature > > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > > not 2.5.x). This feature branch introduces the pluggable > > StoreFileTracker interface. > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > > where the HFiles can be found. The DEFAULT implementation of this > > StoreFileTracker (e.g. files in a family's directory) is still the > > default implementation. This merge would introduce a FILE implementation > > which uses a flat-file in each column family to track the files which > > make up this Store. This feature is notable for HBase as it invalidate > > the need for HBOSS (a distributed locking layer in hbase-filesystem) > > when Amazon S3 is used for HBase data. > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > > to merge. > > > > Covering some high-level details/FAQ on this work: > > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies > > using this feature. > > * YCSB (load) indicates a slight performance improvement when using S3 > > as the storage system for HBase as compared to using HBOSS [3] > > * A new section was added to the HBase book which covers the feature and > > how to use it. > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > > which includes things like making user consumption easier and additional > > metrics to measure effectiveness of the feature. > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > > modification (though I do not expect to need the lazy-consensus caveat). > > Please vote: > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > -1: Do not merge these changes because ... > > > > Big thank you to all of the super hard work that Duo, Wellington, and > > Szabolcs have put into this feature. > > > > - Josh > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > [3] > > > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > > >
re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
Thanks to all who took the time to vote already! So far, we have 5 +1's (2 binding, 3 non-binding) which is sufficient for a lazy-consensus RESULT. However, I'd love to see a vote from someone who didn't contribute code to this feature. I know running an in-depth technical analysis is a major undertaking. I'd ask for a third binding vote even if someone just does a high-level review of the work on the feature branch. At the same time, I also know many folks will be relaxing with their families. I'll leave this open for 2 more days in the hopes that some other folks will still have time to weigh in. On 2021/12/16 21:30:09 Josh Elser wrote: Hi! I'm extremely pleased to send this official vote to merge the feature branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). This feature branch introduces the pluggable StoreFileTracker interface. The StoreFileTracker allows the StoreFileEngine to be decoupled from where the HFiles can be found. The DEFAULT implementation of this StoreFileTracker (e.g. files in a family's directory) is still the default implementation. This merge would introduce a FILE implementation which uses a flat-file in each column family to track the files which make up this Store. This feature is notable for HBase as it invalidate the need for HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is used for HBase data. We had a DISCUSS thread [2] in which the overall sentiment was positive to merge. Covering some high-level details/FAQ on this work: * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies using this feature. * YCSB (load) indicates a slight performance improvement when using S3 as the storage system for HBase as compared to using HBOSS [3] * A new section was added to the HBase book which covers the feature and how to use it. * There is some follow-on work expected, tracked in HBASE-26584 [4], which includes things like making user consumption easier and additional metrics to measure effectiveness of the feature. As is customary, this vote will be open for at least 3 days (2021/12/19 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code modification (though I do not expect to need the lazy-consensus caveat). Please vote: +1: Merge the changes from HBASE-26067 to master and branch-2 -1: Do not merge these changes because ... Big thank you to all of the super hard work that Duo, Wellington, and Szabolcs have put into this feature. - Josh [1] https://issues.apache.org/jira/browse/HBASE-26067 [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 [3] https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 [4] https://issues.apache.org/jira/browse/HBASE-26584
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
+1 (non-binding) Regards, Pankaj On Sat, Dec 18, 2021, 2:35 AM Josh Elser wrote: > I guess I forgot to vote :) > > +1 (binding) > > On 12/17/21 8:57 AM, 张铎(Duo Zhang) wrote: > > Big +1 from me. > > > > This is a big step for making HBase more cloud native. > > > > I've already rebased HBASE-26067 to the latest master. > > > > Thanks~ > > > > Ankit Singhal 于2021年12月17日周五 07:17写道： > > > >> +1 > >> > >> On Thu, Dec 16, 2021 at 1:53 PM Andor Molnar wrote: > >> > >>> +1 (non-binding) > >>> > >>> Andor > >>> > >>> > >>> > On 2021. Dec 16., at 22:30, Josh Elser wrote: > > Hi! > > I'm extremely pleased to send this official vote to merge the feature > >>> branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > >> not > >>> 2.5.x). This feature branch introduces the pluggable StoreFileTracker > >>> interface. > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > >>> where the HFiles can be found. The DEFAULT implementation of this > >>> StoreFileTracker (e.g. files in a family's directory) is still the > >> default > >>> implementation. This merge would introduce a FILE implementation which > >> uses > >>> a flat-file in each column family to track the files which make up this > >>> Store. This feature is notable for HBase as it invalidate the need for > >>> HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 > is > >>> used for HBase data. > > We had a DISCUSS thread [2] in which the overall sentiment was > positive > >>> to merge. > > Covering some high-level details/FAQ on this work: > * Wellington and Szabolcs have successfully run ITBLL with Chaos > >> Monkies > >>> using this feature. > * YCSB (load) indicates a slight performance improvement when using S3 > >>> as the storage system for HBase as compared to using HBOSS [3] > * A new section was added to the HBase book which covers the feature > >> and > >>> how to use it. > * There is some follow-on work expected, tracked in HBASE-26584 [4], > >>> which includes things like making user consumption easier and > additional > >>> metrics to measure effectiveness of the feature. > > As is customary, this vote will be open for at least 3 days > (2021/12/19 > >>> 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > >>> modification (though I do not expect to need the lazy-consensus > caveat). > >>> Please vote: > > +1: Merge the changes from HBASE-26067 to master and branch-2 > -1: Do not merge these changes because ... > > Big thank you to all of the super hard work that Duo, Wellington, and > >>> Szabolcs have put into this feature. > > - Josh > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > [3] > >>> > >> > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > [4] https://issues.apache.org/jira/browse/HBASE-26584 > >>> > >>> > >> > > >
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
I guess I forgot to vote :) +1 (binding) On 12/17/21 8:57 AM, 张铎(Duo Zhang) wrote: Big +1 from me. This is a big step for making HBase more cloud native. I've already rebased HBASE-26067 to the latest master. Thanks~ Ankit Singhal 于2021年12月17日周五 07:17写道： +1 On Thu, Dec 16, 2021 at 1:53 PM Andor Molnar wrote: +1 (non-binding) Andor On 2021. Dec 16., at 22:30, Josh Elser wrote: Hi! I'm extremely pleased to send this official vote to merge the feature branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). This feature branch introduces the pluggable StoreFileTracker interface. The StoreFileTracker allows the StoreFileEngine to be decoupled from where the HFiles can be found. The DEFAULT implementation of this StoreFileTracker (e.g. files in a family's directory) is still the default implementation. This merge would introduce a FILE implementation which uses a flat-file in each column family to track the files which make up this Store. This feature is notable for HBase as it invalidate the need for HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is used for HBase data. We had a DISCUSS thread [2] in which the overall sentiment was positive to merge. Covering some high-level details/FAQ on this work: * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies using this feature. * YCSB (load) indicates a slight performance improvement when using S3 as the storage system for HBase as compared to using HBOSS [3] * A new section was added to the HBase book which covers the feature and how to use it. * There is some follow-on work expected, tracked in HBASE-26584 [4], which includes things like making user consumption easier and additional metrics to measure effectiveness of the feature. As is customary, this vote will be open for at least 3 days (2021/12/19 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code modification (though I do not expect to need the lazy-consensus caveat). Please vote: +1: Merge the changes from HBASE-26067 to master and branch-2 -1: Do not merge these changes because ... Big thank you to all of the super hard work that Duo, Wellington, and Szabolcs have put into this feature. - Josh [1] https://issues.apache.org/jira/browse/HBASE-26067 [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 [3] https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 [4] https://issues.apache.org/jira/browse/HBASE-26584
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
Big +1 from me. This is a big step for making HBase more cloud native. I've already rebased HBASE-26067 to the latest master. Thanks~ Ankit Singhal 于2021年12月17日周五 07:17写道： > +1 > > On Thu, Dec 16, 2021 at 1:53 PM Andor Molnar wrote: > > > +1 (non-binding) > > > > Andor > > > > > > > > > On 2021. Dec 16., at 22:30, Josh Elser wrote: > > > > > > Hi! > > > > > > I'm extremely pleased to send this official vote to merge the feature > > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, > not > > 2.5.x). This feature branch introduces the pluggable StoreFileTracker > > interface. > > > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > > where the HFiles can be found. The DEFAULT implementation of this > > StoreFileTracker (e.g. files in a family's directory) is still the > default > > implementation. This merge would introduce a FILE implementation which > uses > > a flat-file in each column family to track the files which make up this > > Store. This feature is notable for HBase as it invalidate the need for > > HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is > > used for HBase data. > > > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > > to merge. > > > > > > Covering some high-level details/FAQ on this work: > > > * Wellington and Szabolcs have successfully run ITBLL with Chaos > Monkies > > using this feature. > > > * YCSB (load) indicates a slight performance improvement when using S3 > > as the storage system for HBase as compared to using HBOSS [3] > > > * A new section was added to the HBase book which covers the feature > and > > how to use it. > > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > > which includes things like making user consumption easier and additional > > metrics to measure effectiveness of the feature. > > > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > > modification (though I do not expect to need the lazy-consensus caveat). > > Please vote: > > > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > > -1: Do not merge these changes because ... > > > > > > Big thank you to all of the super hard work that Duo, Wellington, and > > Szabolcs have put into this feature. > > > > > > - Josh > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > > [3] > > > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > > > > >
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
+1 On Thu, Dec 16, 2021 at 1:53 PM Andor Molnar wrote: > +1 (non-binding) > > Andor > > > > > On 2021. Dec 16., at 22:30, Josh Elser wrote: > > > > Hi! > > > > I'm extremely pleased to send this official vote to merge the feature > branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not > 2.5.x). This feature branch introduces the pluggable StoreFileTracker > interface. > > > > The StoreFileTracker allows the StoreFileEngine to be decoupled from > where the HFiles can be found. The DEFAULT implementation of this > StoreFileTracker (e.g. files in a family's directory) is still the default > implementation. This merge would introduce a FILE implementation which uses > a flat-file in each column family to track the files which make up this > Store. This feature is notable for HBase as it invalidate the need for > HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is > used for HBase data. > > > > We had a DISCUSS thread [2] in which the overall sentiment was positive > to merge. > > > > Covering some high-level details/FAQ on this work: > > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies > using this feature. > > * YCSB (load) indicates a slight performance improvement when using S3 > as the storage system for HBase as compared to using HBOSS [3] > > * A new section was added to the HBase book which covers the feature and > how to use it. > > * There is some follow-on work expected, tracked in HBASE-26584 [4], > which includes things like making user consumption easier and additional > metrics to measure effectiveness of the feature. > > > > As is customary, this vote will be open for at least 3 days (2021/12/19 > 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code > modification (though I do not expect to need the lazy-consensus caveat). > Please vote: > > > > +1: Merge the changes from HBASE-26067 to master and branch-2 > > -1: Do not merge these changes because ... > > > > Big thank you to all of the super hard work that Duo, Wellington, and > Szabolcs have put into this feature. > > > > - Josh > > > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > > [3] > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > > [4] https://issues.apache.org/jira/browse/HBASE-26584 > >
Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
+1 (non-binding) Andor > On 2021. Dec 16., at 22:30, Josh Elser wrote: > > Hi! > > I'm extremely pleased to send this official vote to merge the feature branch > for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). > This feature branch introduces the pluggable StoreFileTracker interface. > > The StoreFileTracker allows the StoreFileEngine to be decoupled from where > the HFiles can be found. The DEFAULT implementation of this StoreFileTracker > (e.g. files in a family's directory) is still the default implementation. > This merge would introduce a FILE implementation which uses a flat-file in > each column family to track the files which make up this Store. This feature > is notable for HBase as it invalidate the need for HBOSS (a distributed > locking layer in hbase-filesystem) when Amazon S3 is used for HBase data. > > We had a DISCUSS thread [2] in which the overall sentiment was positive to > merge. > > Covering some high-level details/FAQ on this work: > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies > using this feature. > * YCSB (load) indicates a slight performance improvement when using S3 as the > storage system for HBase as compared to using HBOSS [3] > * A new section was added to the HBase book which covers the feature and how > to use it. > * There is some follow-on work expected, tracked in HBASE-26584 [4], which > includes things like making user consumption easier and additional metrics to > measure effectiveness of the feature. > > As is customary, this vote will be open for at least 3 days (2021/12/19 2130 > GMT). We'll follow the standard ASF lazy-consensus rules for code > modification (though I do not expect to need the lazy-consensus caveat). > Please vote: > > +1: Merge the changes from HBASE-26067 to master and branch-2 > -1: Do not merge these changes because ... > > Big thank you to all of the super hard work that Duo, Wellington, and > Szabolcs have put into this feature. > > - Josh > > [1] https://issues.apache.org/jira/browse/HBASE-26067 > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 > [3] > https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 > [4] https://issues.apache.org/jira/browse/HBASE-26584
[VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2
Hi! I'm extremely pleased to send this official vote to merge the feature branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). This feature branch introduces the pluggable StoreFileTracker interface. The StoreFileTracker allows the StoreFileEngine to be decoupled from where the HFiles can be found. The DEFAULT implementation of this StoreFileTracker (e.g. files in a family's directory) is still the default implementation. This merge would introduce a FILE implementation which uses a flat-file in each column family to track the files which make up this Store. This feature is notable for HBase as it invalidate the need for HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is used for HBase data. We had a DISCUSS thread [2] in which the overall sentiment was positive to merge. Covering some high-level details/FAQ on this work: * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies using this feature. * YCSB (load) indicates a slight performance improvement when using S3 as the storage system for HBase as compared to using HBOSS [3] * A new section was added to the HBase book which covers the feature and how to use it. * There is some follow-on work expected, tracked in HBASE-26584 [4], which includes things like making user consumption easier and additional metrics to measure effectiveness of the feature. As is customary, this vote will be open for at least 3 days (2021/12/19 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code modification (though I do not expect to need the lazy-consensus caveat). Please vote: +1: Merge the changes from HBASE-26067 to master and branch-2 -1: Do not merge these changes because ... Big thank you to all of the super hard work that Duo, Wellington, and Szabolcs have put into this feature. - Josh [1] https://issues.apache.org/jira/browse/HBASE-26067 [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1 [3] https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499 [4] https://issues.apache.org/jira/browse/HBASE-26584
