[
https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715060#comment-16715060
]
Josh Elser commented on HBASE-21246:
------------------------------------
{quote}used master for pre-commit jenkin build as HBASE-20952 yet to be rebased
{quote}
Just rebased that branch now.
{code:java}
- PriorityBlockingQueue<Path> newPaths =
- new PriorityBlockingQueue<Path>(queueSizePerGroup, new
LogsComparator());
- pathsLoop: for (Path path : queue) {
- if (fs.exists(path)) { // still in same location, don't need to do
anything
- newPaths.add(path);
+ PriorityBlockingQueue<WALIdentity> newWalIds =
+ new PriorityBlockingQueue<WALIdentity>(queueSizePerGroup, new
LogsComparator());
+ pathsLoop: for (WALIdentity walId : queue) {
+ if (fs.exists(((FSWALIdentity)walId).getPath())) { // still in same
location, don't need to do anything
+ newWalIds.add(walId);{code}
I see RecoveredReplicationSource.java still needs some "unraveling" from Path.
{code:java}
- stat = fs.getFileStatus(this.currentPath);
+ stat =
fs.getFileStatus(((FSWALIdentity)this.currentWAlIdentity).getPath());{code}
{code:java}
- Path archivedLog = getArchivedLog(path);
- if (!path.equals(archivedLog)) {
+ FSWALIdentity archivedLog = new
FSWALIdentity(getArchivedLog(walId.getPath()));
+ if (!walId.equals(archivedLog)) {{code}
WALEntryStream is in a similar position (a couple of others than just those
pulled out above).
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java
index 75439fe6c5..ad9f6bda30 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java
@@ -63,7 +63,7 @@ class DisabledWALProvider implements WALProvider {
if (null == providerId) {
providerId = "defaultDisabled";
}
- disabled = new DisabledWAL(new Path(FSUtils.getWALRootDir(conf),
providerId), conf, null);
+ disabled = new DisabledWAL(new FSWALIdentity(new
Path(FSUtils.getWALRootDir(conf), providerId)), conf, null);{code}
{code:java}
- protected final Path path;
+ protected final FSWALIdentity walId;{code}
Should DisabledWALProvider have its own implementation of WALIdentity? Looks
like we just pass a "special" Path in the FS-based case now – maybe we just
make some special implementation of WALIdentity for it instead.
Overall, I think this is a really nice middle-ground of changing "enough"
without changing too much. As long as we can spin out the above refactorings
into some follow-on work, I would be happy to land this on the feature branch.
> Introduce WALIdentity interface
> -------------------------------
>
> Key: HBASE-21246
> URL: https://issues.apache.org/jira/browse/HBASE-21246
> Project: HBase
> Issue Type: Sub-task
> Reporter: Ted Yu
> Assignee: Ted Yu
> Priority: Major
> Fix For: HBASE-20952
>
> Attachments: 21246.003.patch, 21246.20.txt, 21246.21.txt,
> 21246.23.txt, 21246.24.txt, 21246.25.txt, 21246.26.txt, 21246.34.txt,
> 21246.37.txt, 21246.39.txt, 21246.41.txt, 21246.43.txt,
> 21246.HBASE-20952.001.patch, 21246.HBASE-20952.002.patch,
> 21246.HBASE-20952.004.patch, 21246.HBASE-20952.005.patch,
> 21246.HBASE-20952.007.patch, 21246.HBASE-20952.008.patch,
> HBASE-21246.master.001.patch, replication-src-creates-wal-reader.jpg,
> wal-factory-providers.png, wal-providers.png, wal-splitter-reader.jpg,
> wal-splitter-writer.jpg
>
>
> We are introducing WALIdentity interface so that the WAL representation can
> be decoupled from distributed filesystem.
> The interface provides getName method whose return value can represent
> filename in distributed filesystem environment or, the name of the stream
> when the WAL is backed by log stream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
