[
https://issues.apache.org/jira/browse/HBASE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-10378:
--------------------------------
Release Note:
HBase internals for the write ahead log have been refactored. Advanced users of
HBase should be aware of the following changes.
Public Audience
- The command for analyzing write ahead logs has been renamed from 'hlog' to
'wal'. The old usage is deprecated and will be removed in a future version.
- Some utility methods in the HBaseTesetingUtility related to testing
write-ahead-logs were changed in incompatible ways. No functionality has been
removed, but method names and arguments have changed. See the
HBaseTestingUtility javadoc for details.
- The HLogInputFormat utility class for processing logs with MapReduce has
been deprecated and will be removed in a future version. Users should switch to
the WALInputFormat.
- The labeling of server metrics on the region server status pages changed.
Previously, the number of backing files for the write ahead log was labeled
'Num. HLog Files'. If you wish to see this statistic now, please look for the
label 'Num. WAL Files.' If you rely on JMX for these metrics, their location
has not changed.
LimitedPrivate(COPROC) Audience, LimitedPrivate(PHOENIX)
- The RegionObserver API has been updated. The changes are both binary and
source backwards compatible for coprocessors that use the BaseRegionObserver
class. For those that implement RegionObserver directly the changes are binary
backwards compatible. Depending on the internals of future HBase versions,
coprocessors using the deprecated API may not see all WAL related events. Users
are strongly encouraged to update their use of the API; see the RegionObserver
javadoc for details.
- Classes related to reading WAL entries (ReaderBase, ProtobufLogReader,
SequenceFileLogReader) have changed in a backwards incompatible way. Users who
referenced HLog.Reader directly or HLog.Entry will have to update. These
changes do not impact compatibility with extant wal files.
- The WALObserver API has been updated. The changes are both binary and
source backwards compatible for coprocessors that use the BaseWALObserver
class. For those that implement WALObserver directly the changes are binary
backwards compatible. Depending on the internals of future HBase versions,
coprocessors using the deprecated API may not see all WAL related events. Users
are strongly encouraged to update their use of the API; see the WALObserver
javadoc for details.
- The WALCoprocessorEnvironment has changed in a backwards incompatible way.
WALObserver coprocessors that relied on retrieving an object representing the
write ahead log instance will have to be updated.
LimitedPrivate(REPLICATION) Audience
- The WALEntryFilter API has changed in a backwards incompatible way.
Implementers will have to be updated.
- The ReplicationEndpoint.ReplicateContext API has changed in a backwards
incompatible way. Implementers who use this interface will have to be updated.
These changes do not impact wire compatibility for replicating between clusters.
- The HLogKey API is deprecated in favor of the WALKey API. Additionally, the
HLogKey API has changed in a backwards incompatible way by changing from
implementing WriteableComparable<HLogKey> to implementing Writeable and
Comparable<WALKey>.
was:
HBase internals for the write ahead log have been refactored. Advanced users of
HBase should be aware of the following changes.
- The command for analyzing write ahead logs has been renamed from 'hlog' to
'wal'. The old usage is deprecated and will be removed in a future version.
- Some utility methods in the HBaseTesetingUtility related to testing
write-ahead-logs were changed in incompatible ways. No functionality has been
removed, but method names and arguments have changed. See the javadoc for
HBaseTestingUtility for details.
- The labeling of server metrics on the region server status pages changed.
Previously, the number of backing files for the write ahead log was labeled
'Num. HLog Files'. If you wish to see this statistic now, please look for the
label 'Num. WAL Files.' If you rely on JMX for these metrics, their location
has not changed.
Updating the release notes based on RB feedback and the current (yet to be
posted) reworking of the code to maintain more backwards compatibility.
I believe at this point there aren't any backwards incompatibilities with
Phoenix's non-test code. There is impact in both unit and integration tests.
The replication code is further into the guts of the write-ahead-log than the
coprocessors were, so it has changed more. Projects that rely on hooking into
that code path will be broken. This includes the only prominent example I could
think of, the HBase Lily Indexer. Since they have a compatibility layer
already, these changes should just mean that they need an updated one based on
their current 0.98 shim that accounts for the class renames in this patch.
Open to suggestions on other downstream impacts I should look for.
> Divide HLog interface into User and Implementor specific interfaces
> -------------------------------------------------------------------
>
> Key: HBASE-10378
> URL: https://issues.apache.org/jira/browse/HBASE-10378
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Himanshu Vashishtha
> Assignee: Sean Busbey
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 10378-1.patch, 10378-2.patch
>
>
> HBASE-5937 introduces the HLog interface as a first step to support multiple
> WAL implementations. This interface is a good start, but has some
> limitations/drawbacks in its current state, such as:
> 1) There is no clear distinction b/w User and Implementor APIs, and it
> provides APIs both for WAL users (append, sync, etc) and also WAL
> implementors (Reader/Writer interfaces, etc). There are APIs which are very
> much implementation specific (getFileNum, etc) and a user such as a
> RegionServer shouldn't know about it.
> 2) There are about 14 methods in FSHLog which are not present in HLog
> interface but are used at several places in the unit test code. These tests
> typecast HLog to FSHLog, which makes it very difficult to test multiple WAL
> implementations without doing some ugly checks.
> I'd like to propose some changes in HLog interface that would ease the multi
> WAL story:
> 1) Have two interfaces WAL and WALService. WAL provides APIs for
> implementors. WALService provides APIs for users (such as RegionServer).
> 2) A skeleton implementation of the above two interface as the base class for
> other WAL implementations (AbstractWAL). It provides required fields for all
> subclasses (fs, conf, log dir, etc). Make a minimal set of test only methods
> and add this set in AbstractWAL.
> 3) HLogFactory returns a WALService reference when creating a WAL instance;
> if a user need to access impl specific APIs (there are unit tests which get
> WAL from a HRegionServer and then call impl specific APIs), use AbstractWAL
> type casting,
> 4) Make TestHLog abstract and let all implementors provide their respective
> test class which extends TestHLog (TestFSHLog, for example).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
