[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609868#comment-16609868
]
Josh Elser commented on HBASE-20952:
------------------------------------
bq. the study of other logging systems resulted in "...no significant influence
on HBase WAL API design." though above Josh says " I can say that a significant
portion of the direction was strongly influenced by Apache DistributedLog".
I figure I should handle this one directly: to be honest, I shouldn't have
dropped this comment here in HBase. That one is relevant for the Ratis
LogService API. In that lengthy design doc that I put up and worked with a
number of you on, it was a primary goal to design the HBase WAL API *only* to
what HBase needs. This was to avoid the trap of trading one hard-dependency
(HDFS) for another.
There was lots of chatter over on RATIS-272 (for those who want to be a part of
that), and we got an initial API from that. The first wag at how we think the
LogService should be implemented mimics the structure of how DistributedLog
does things, which ultimately pushes us towards an API which looks like DL.
Granted, I think what we have is more clean than DL, but that's where my
comment came from.
I'd be remiss if I didn't at least acknowledge: of course we want to be aware
of how general distributed log (the concept, not the project) APIs look like
(otherwise, how do we know if what we're building would work generically).
However, we still want to approach this initially by understanding what the
requirements are inside of HBase, and then finding a happy medium between the
systems we know we want to support.
> Re-visit the WAL API
> --------------------
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Josh Elser
> Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
