Filed HBASE-25988. And also posted a simple design doc there.
https://docs.google.com/document/d/1Ao_jwa3ekXghAKXVpO7fbX_BXrTv3-RgxsneNULRXx0/edit?usp=sharing Wellington Chevreuil <wellington.chevre...@gmail.com> 于2021年6月9日周三 下午6:28写道： > I say go for it. Can you create it as a subtask under the > umbrella HBASE-24749 and PR to that feature branch? I had been working on > the tasks related to avoid "creating files in temp then rename" for flushes > and compactions (HBASE-25391 and HBASE-25392), had managed to successfully > run it with the PersistedStoreEngine from HBASE-25395. > > Em ter., 8 de jun. de 2021 às 08:02, 张铎(Duo Zhang) <palomino...@gmail.com> > escreveu: > > > Can not comment on the google doc so just reply here. > > > > Recently I read a bit of Hudi and Iceberg, they both use files on the > > filesystem for committing changes by default, so you can write to the > data > > directory directly without worrying about mess up the data files. > > > > If no big concerns, I could start a implement a POC by going with the > plain > > file approach. > > > > Thanks. > > > > Josh Elser <els...@apache.org> 于2021年5月27日周四 上午3:21写道： > > > > > Thanks Stack! (access given, as google probably told you already). > > > > > > Please keep me honest. > > > > > > On 5/26/21 12:29 PM, Stack wrote: > > > > And, what is there currently is a nice write-up.... > > > > S > > > > > > > > On Wed, May 26, 2021 at 9:26 AM Stack <st...@duboce.net> wrote: > > > > > > > >> Can I have comment access please Josh? > > > >> S > > > >> > > > >> On Tue, May 25, 2021 at 8:24 PM Josh Elser <els...@apache.org> > wrote: > > > >> > > > >>> Hi folks, > > > >>> > > > >>> This is a follow-on for the HBASE-24749 discussion on storefile > > > >>> tracking, specifically focusing on where/how do we store the list > of > > > >>> files for each Store. > > > >>> > > > >>> I tried to capture my thoughts and the suggestions by Duo and > > > Wellington > > > >>> in this google doc [1]. > > > >>> > > > >>> Please feel free to ask for edit permission (and send me a note if > > your > > > >>> email address isn't one that I would otherwise recognize :) ) to > > > >>> correct, improve, or expand on any other sections. > > > >>> > > > >>> FWIW, I was initially not super excited about a per-Store file, > but, > > > the > > > >>> more I think about it, the more I'm coming around to that idea. I > > think > > > >>> it will be more "exception-handling", but avoid the long-term > > > >>> operational burden of yet-another-important-system-table. > > > >>> > > > >>> - Josh > > > >>> > > > >>> [1] > > > >>> > > > >>> > > > > > > https://docs.google.com/document/d/1yzjvQvQfnT-M8ZgKdcQNedF8HssTnQR2loPkZtlJGVg/edit?usp=sharing > > > >>> > > > >> > > > > > > > > > >
