VladRodionov commented on code in PR #8327:
URL: https://github.com/apache/hbase/pull/8327#discussion_r3384424923
##########
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALInputFormat.java:
##########
@@ -120,4 +122,44 @@ public void testHandlesArchivedWALFiles() throws Exception
{
assertEquals(archiveWal.toString(), split.getLogFileName());
}
+ /**
+ * Test that an empty WAL file (which causes WALHeaderEOFException) is
gracefully handled and
+ * skipped rather than causing the job to fail.
+ */
+ @Test
+ public void testHandlesEmptyWALFile() throws Exception {
+ Configuration conf = TEST_UTIL.getConfiguration();
+
+ // Create an empty WAL file
+ Path walRootDir = CommonFSUtils.getWALRootDir(conf);
+ Path emptyWalFile =
+ new Path(walRootDir, "WALs/empty-wal-test/empty." +
EnvironmentEdgeManager.currentTime());
+ TEST_UTIL.getTestFileSystem().mkdirs(emptyWalFile.getParent());
+ TEST_UTIL.getTestFileSystem().create(emptyWalFile).close();
+
+ JobContext ctx = Mockito.mock(JobContext.class);
+ conf.set(FileInputFormat.INPUT_DIR, emptyWalFile.toString());
+ conf.set(WALPlayer.INPUT_FILES_SEPARATOR_KEY, ";");
+ Mockito.when(ctx.getConfiguration()).thenReturn(conf);
+ Job job = Job.getInstance(conf);
+ TableMapReduceUtil.initCredentialsForCluster(job, conf);
+ Mockito.when(ctx.getCredentials()).thenReturn(job.getCredentials());
+
+ // Create record reader and verify it handles the empty file gracefully
+ try (WALInputFormat.WALKeyRecordReader reader = new
WALInputFormat.WALKeyRecordReader()) {
+ TaskAttemptContext taskCtx = Mockito.mock(TaskAttemptContext.class);
+ Mockito.when(taskCtx.getConfiguration()).thenReturn(conf);
+
+ WALInputFormat wif = new WALInputFormat();
+ List<InputSplit> splits = wif.getSplits(ctx);
+ assertEquals(1, splits.size());
+ WALInputFormat.WALSplit split = (WALInputFormat.WALSplit) splits.get(0);
+
+ // This should not throw WALHeaderEOFException - it should return false
for nextKeyValue()
+ reader.initialize(split, taskCtx);
+ // nextKeyValue() should return false since the file is empty (reader is
null)
+ assertFalse(reader.nextKeyValue());
+ }
Review Comment:
nit: Could we delete emptyWalFile in a finally block? The test creates a WAL
file under the mini-cluster WAL root and does not appear to clean it up. Even
if the test cluster cleanup eventually removes it, local cleanup would make the
test more isolated.
Otherwise, lgtm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
