(orc) branch branch-2.0 updated: ORC-1630: Test using VectoredIO of hadoop to read ORC

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/orc.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
     new bca54a84a ORC-1630: Test using VectoredIO of hadoop to read ORC
bca54a84a is described below

commit bca54a84a566b521542930e8a6f82483941bd041
Author: sychen <syc...@ctrip.com>
AuthorDate: Tue Feb 20 20:01:17 2024 -0800

    ORC-1630: Test using VectoredIO of hadoop to read ORC
    
    ### What changes were proposed in this pull request?
    This PR aims to test reading ORC using VectoredIO of hadoop.
    
    ### Why are the changes needed?
    ORC-1251 supports reading using VectoredIO, but no test coverage.
    
    ### How was this patch tested?
    Add UT
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #1814 from cxzl25/ORC-1630.
    
    Authored-by: sychen <syc...@ctrip.com>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
    (cherry picked from commit 39368b5910032e0dbb257e5ebc98061e50a5e1e1)
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 .../org/apache/orc/impl/TestRecordReaderImpl.java   | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java 
b/java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java
index c0ff4f7e8..f0124715b 100644
--- a/java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java
+++ b/java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java
@@ -97,6 +97,7 @@ import static org.mockito.Mockito.atLeastOnce;
 import static org.mockito.Mockito.doNothing;
 import static org.mockito.Mockito.doThrow;
 import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.times;
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
@@ -2711,4 +2712,24 @@ public class TestRecordReaderImpl {
       base += batch.size;
     }
   }
+
+  @Test
+  public void testHadoopVectoredIO() throws Exception {
+    Configuration conf = new Configuration();
+    Path filePath = new 
Path(TestVectorOrcFile.getFileFromClasspath("orc-file-11-format.orc"));
+
+    FileSystem localFileSystem = FileSystem.getLocal(conf);
+    FSDataInputStream fsDataInputStream = localFileSystem.open(filePath);
+
+    FileSystem spyLocalFileSystem = spy(localFileSystem);
+    FSDataInputStream spyFSDataInputStream = spy(fsDataInputStream);
+    when(spyLocalFileSystem.open(filePath)).thenReturn(spyFSDataInputStream);
+
+    Reader reader = OrcFile.createReader(filePath,
+        OrcFile.readerOptions(conf).filesystem(spyLocalFileSystem));
+    RecordReader recordReader = reader.rows(reader.options());
+    recordReader.close();
+
+    verify(spyFSDataInputStream, atLeastOnce()).readVectored(any(), any());
+  }
 }