[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604701#comment-16604701
]
stack commented on HBASE-20952:
-------------------------------
On the writeup attached to an rb textbox
* Should be a doc. attached here to this issue. It'll be lost if just a
comment in rb.
* The first sentence is just wrong .... "This patch aims to refactor WAL
related classes in order to facilitate introduction of a non-FS based WAL
implementation." Ratis doesn't have an FS? The WAL will hang on skyhooks? When
the target is so inspecifically described, how we have a hope of knowing when
we've hit it?
* A sentence like, "Before making code changes, we studied Apache Kafka,
Apache BookKeeper and Apache Ratis.", is usually followed by a summary of what
was learned.
* "Some design considerations:" begs the question of what design questions are
not listed.
* This is incorrect too... right: "The refactoring of WAL related code is to
decouple WAL from FileSystem so that other consensus protocols can be
accommodated to back different WAL implementations." BooKeeper, for instance, a
purported target, is not consensus based... nor Kafka.
* This is confusing => "metadata provider, FS based and non-FS based, provides
metadata for given Id (represented by WALInfo) of any particular WAL" ... 'Id'
is introduced w/o explanation. WALInfo represents 'Id' but is called WALInfo?
* Why is a class that identifies a WAL called WALInfo and not WALIdentifier or
WAL_ID? "WALInfo : identification of any particular WAL" ... Info is effete.
* Why so many classes FSWALInfo, AbstractWAL, then AbstractFSWAL ... ? And why
we even have an FS version of anything? I'd think we'd have ClassicWAL,
KafkaWAL, rather than an attempt at a generic "FSWAL"...
* There is a AbstractWALEntryStream.... How many implementations has it be used
with? Classic HBase WAL and?
* What is this? "FSRecoveredReplicationSource : FS based implementation for
RecoveredReplicationSource" And we just have an FS based one?
* Why is it ListWal but everywhere else WAL is capitalized?
Where is discussion of how to use the new API?
Where do I even go to look for the new WAL API? Presuming WAL is the place,
looking at it, the only change is removal of roll writer? So now the
implmentation is responsible for rolling? Even when trouble syncing? Can we
have discussion on how this will be in the new implementation along w/ any
other change in mechanics?
Thats enough for now.
> Re-visit the WAL API
> --------------------
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Josh Elser
> Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
