[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615326#comment-16615326
]
Sergey Soldatov commented on HBASE-20952:
-----------------------------------------
A few of my personal thoughts that are based on previous experience of
implementing such kind of functionality in HBase (C5, non-stop hbase,
hydrabase). It would be better to perform atomic meaningful changes that
address a particular problem without breaking the existing functionality and
without adding an additional complexity. We need a 'pluggable' WALProvider
that will be agnostic to file system structure. Fine. Let's make it
configurable and remove the dependency on file systems (introducing WALIdentity
or whatever we call it). It doesn't work with the current implementation of
replication? Not good, but we may mention that it in the Limitation section.
All we need is an understanding that it's possible to implement even if it
would require a lot of efforts. Possible the replication itself requires a new
Jira 'Revisit Replication approach' especially if we are talking about
different kind of wal providers. Possible someone would love to have a setup
with replication between on-prem cluster that is using regular WAL and another
in AWS using Bookeeper wal. And that would require another layer of abstraction
for replication itself. Should it be done under this jira? Honestly speaking I
don't think so.
> Re-visit the WAL API
> --------------------
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Josh Elser
> Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
