Can not comment on the google doc so just reply here. Recently I read a bit of Hudi and Iceberg, they both use files on the filesystem for committing changes by default, so you can write to the data directory directly without worrying about mess up the data files.
If no big concerns, I could start a implement a POC by going with the plain file approach. Thanks. Josh Elser <els...@apache.org> 于2021年5月27日周四 上午3:21写道： > Thanks Stack! (access given, as google probably told you already). > > Please keep me honest. > > On 5/26/21 12:29 PM, Stack wrote: > > And, what is there currently is a nice write-up.... > > S > > > > On Wed, May 26, 2021 at 9:26 AM Stack <st...@duboce.net> wrote: > > > >> Can I have comment access please Josh? > >> S > >> > >> On Tue, May 25, 2021 at 8:24 PM Josh Elser <els...@apache.org> wrote: > >> > >>> Hi folks, > >>> > >>> This is a follow-on for the HBASE-24749 discussion on storefile > >>> tracking, specifically focusing on where/how do we store the list of > >>> files for each Store. > >>> > >>> I tried to capture my thoughts and the suggestions by Duo and > Wellington > >>> in this google doc [1]. > >>> > >>> Please feel free to ask for edit permission (and send me a note if your > >>> email address isn't one that I would otherwise recognize :) ) to > >>> correct, improve, or expand on any other sections. > >>> > >>> FWIW, I was initially not super excited about a per-Store file, but, > the > >>> more I think about it, the more I'm coming around to that idea. I think > >>> it will be more "exception-handling", but avoid the long-term > >>> operational burden of yet-another-important-system-table. > >>> > >>> - Josh > >>> > >>> [1] > >>> > >>> > https://docs.google.com/document/d/1yzjvQvQfnT-M8ZgKdcQNedF8HssTnQR2loPkZtlJGVg/edit?usp=sharing > >>> > >> > > >
