Apache9 commented on a change in pull request #774: HBASE-23221 Polish the WAL
interface after HBASE-23181
URL: https://github.com/apache/hbase/pull/774#discussion_r340398560
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALEdit.java
##########
@@ -52,45 +49,104 @@
* single record, in PB format, followed (optionally) by Cells written via the
WALCellEncoder.
* <p>This class is LimitedPrivate for CPs to read-only. The {@link #add}
methods are
* classified as private methods, not for use by CPs.</p>
+ *
+ * <p>A particular WALEdit 'type' is the 'meta' type used to mark key
operational
+ * events in the WAL such as compaction, flush, or region open. These meta
types do not traverse
+ * hbase memstores. They are edits made by the hbase system rather than edit
data submitted by
+ * clients. They only show in the WAL. These 'Meta' types have not been
formally specified
+ * (or made into an explicit class type). They evolved organically. HBASE-8457
suggests codifying
+ * a WALEdit 'type' by adding a type field to WALEdit that gets serialized
into the WAL. TODO.
+ * Would have to work on the consumption-side. Reading WALs on replay we seem
to consume
+ * a Cell-at-a-time rather than by WALEdit. We are already in the below going
out of our
+ * way to figure particular types -- e.g. if a compaction, replay, or close
meta Marker -- during
+ * normal processing so would make sense to do this. Current system is an
awkward marking of Cell
+ * columnfamily as {@link #METAFAMILY} and then setting qualifier based off
meta edit type. For
+ * replay-time where we read Cell-at-a-time, there are utility methods below
for figuring
+ * meta type. See also
+ * {@link #createBulkLoadEvent(RegionInfo, WALProtos.BulkLoadDescriptor)},
+ * {@link #createCompaction(RegionInfo, CompactionDescriptor)}, etc., for
where we create
+ * meta WALEdit instances.</p>
+ *
* <p>WALEdit will accumulate a Set of all column family names referenced by
the Cells
* {@link #add(Cell)}'d. This is an optimization. Usually when loading a
WALEdit, we have the
* column family name to-hand.. just shove it into the WALEdit if available.
Doing this, we can
* save on a parse of each Cell to figure column family down the line when we
go to add the
* WALEdit to the WAL file. See the hand-off in FSWALEntry Constructor.
+ * @see WALKey
*/
// TODO: Do not expose this class to Coprocessors. It has set methods. A CP
might meddle.
@InterfaceAudience.LimitedPrivate({ HBaseInterfaceAudience.REPLICATION,
HBaseInterfaceAudience.COPROC })
public class WALEdit implements HeapSize {
- private static final Logger LOG = LoggerFactory.getLogger(WALEdit.class);
-
- // TODO: Get rid of this; see HBASE-8457
+ // Below defines are for writing WALEdit 'meta' Cells..
+ // TODO: Get rid of this system of special 'meta' Cells. See HBASE-8457. It
suggests
+ // adding a type to WALEdit itself for use denoting meta Edits and their
types.
public static final byte [] METAFAMILY = Bytes.toBytes("METAFAMILY");
- @VisibleForTesting
+
+ /**
+ * @deprecated Since 2.3.0. Not used.
+ */
+ @Deprecated
public static final byte [] METAROW = Bytes.toBytes("METAROW");
+
+ /**
+ * @deprecated Since 2.3.0. Make it protected, internal-use only. Use
+ * {@link #isCompactionMarker(Cell)}
+ */
+ @Deprecated
@VisibleForTesting
public static final byte[] COMPACTION = Bytes.toBytes("HBASE::COMPACTION");
+
+ /**
+ * @deprecated Since 2.3.0. Make it protected, internal-use only.
+ */
+ @Deprecated
@VisibleForTesting
public static final byte [] FLUSH = Bytes.toBytes("HBASE::FLUSH");
- @VisibleForTesting
- public static final byte [] REGION_EVENT =
Bytes.toBytes("HBASE::REGION_EVENT");
+
+ /**
+ * Qualifier for region event meta 'Marker' WALEdits start with the
+ * {@link #REGION_EVENT_PREFIX} prefix ('HBASE::REGION_EVENT::'). After the
prefix,
+ * we note the type of the event which we get from the RegionEventDescriptor
protobuf
+ * instance type (A RegionEventDescriptor protobuf instance is written as
the meta Marker
+ * Cell value). Adding a type suffix means we do not have to deserialize the
protobuf to
+ * figure out what type of event this is.. .just read the qualifier suffix.
For example,
+ * a close region event descriptor will have a qualifier of
HBASE::REGION_EVENT::REGION_CLOSE.
+ * See WAL.proto and the EventType in RegionEventDescriptor protos for all
possible
+ * event types.
+ */
+ private static final String REGION_EVENT_STR = "HBASE::REGION_EVENT";
+ private static final String REGION_EVENT_PREFIX_STR = REGION_EVENT_STR +
"::";
+ private static final byte [] REGION_EVENT_PREFIX =
Bytes.toBytes(REGION_EVENT_PREFIX_STR);
+
+ /**
+ * @deprecated Since 2.3.0. Remove. Not for external use. Not used.
+ */
+ @Deprecated
+ public static final byte [] REGION_EVENT = Bytes.toBytes(REGION_EVENT_STR);
+
+ /**
+ * We use this define figuring if we are carrying a close event.
+ */
+ private static final byte [] REGION_EVENT_CLOSE =
Review comment:
OK so this is the trick... Seems fine?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
