[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604948#comment-16604948
]
stack commented on HBASE-20952:
-------------------------------
As asked-for above, we need a write-up on how this work goes against the
high-level design including learnings, decisions (why), cons, and todos. Asking
us to deduce this info, the model, and intent, by trying to parse a four-pager
patch is a lot to ask. The rb text doesn't satisfy my request above nor yours
that followed after mine. For an example of what I'm after, see the write-up on
HBASE-20708
(https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#),
a doc. I've since gone back to a few times trying to ensure that any changes
to the affected area align w/ the architects' intent.
On your comments:
* On the lead-off, thanks for starting to fill in what should have been there
but the point was an imprecise lead-of colors the reading of what follows.
* If the implementation is informed by "Apache DistributedLog", I'd like to
learn how. Do I have to read DL code and then this patch and compare to figure
it? How I figure if DL namespace'ing was mapped to hbase regions or not? Any
other learnings from this review of the outstanding literature?
* "I think Ted was just trying to capture this." <= The comment is about
consensus based WAL implementations... Which is an exclusive club. IIRC, the
high-level design was more ecumenical. Are we dropping target impls?
* "Which do you think is better?" <= Either.
* "I'm struggling to peel away a suggestion from the criticism." <= This area
of the codebase has seen heavy accretion. It is overladen w/ Interface,
Abstracts, Impls, and Utils. I see an opportunity for nice cleanup via
application of a strong design. From review of the commentary (only!), it looks
like we are being freighted with a new set or Infos, Abstracts, and Interfaces,
with an "FS" dimension thrown-in. On "Provider" being a core concept already,
I'd forgotten. A write-up that talked up how existing Provider notion is being
doubled-down on would help with navigation.
* "A smaller review in which only the new WAL API is provided?" <= I'd think
this is how we'd lead off? In a write-up. The shiny new WAL interface that has
had all FS/Path/File purged from it. Tah-dah!
* "I think we need to tease this apart." <= Does the patch say this anywhere?
This is a TODO? Follow-on? No sense from the rb comment.
* "Do you have something in mind in how this abstraction should work?" <= I
have ideas but ideas are cheap ("API has an append that returns a sequence id.
Also has a replay that takes an sequenceid and it gives you back all edits at
and after the sequence id. Ideally, could do sequence id by region or group of
regions and ask for replay by group or region."). I defer to you fellows that
are doing the hard work. I'd like to benefit from what you've learned though
from reviewing existing implementations. I don't think I will get this reading
a fat patch only.
> Re-visit the WAL API
> --------------------
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Josh Elser
> Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
