[
https://issues.apache.org/jira/browse/TRAFODION-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530588#comment-16530588
]
ASF GitHub Bot commented on TRAFODION-3126:
-------------------------------------------
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1626#discussion_r199650506
--- Diff: core/sql/src/main/java/org/trafodion/sql/HDFSClient.java ---
@@ -142,21 +154,32 @@
HDFSRead()
{
}
-
+
public Object call() throws IOException
{
int bytesRead;
int totalBytesRead = 0;
if (compressed_) {
bufArray_ = new byte[ioByteArraySizeInKB_ * 1024];
- } else
- if (! buf_.hasArray()) {
- try {
- fsdis_.seek(pos_);
- } catch (EOFException e) {
- isEOF_ = 1;
- return new Integer(totalBytesRead);
- }
+ }
+ else {
+ // alluxio doesn't support direct ByteBuffer reads
+ // Hence, create a non-direct ByteBuffer, read into
+ // byteArray backing up this ByteBuffer and
+ // then copy the data read to direct ByteBuffer for the
+ // native layer to process the data
+ if ((! alluxioNotInstalled_) && fs_ instanceof
alluxio.hadoop.FileSystem) {
+ savedBuf_ = buf_;
+ buf_ = ByteBuffer.allocate(savedBuf_.capacity());
--- End diff --
Do we do this allocation every time we call the call() method? If so, could
we avoid that and do this only once? The same applies to line 163 above.
> Refactored HDFS client implementation should also support Alluxio file system
> -----------------------------------------------------------------------------
>
> Key: TRAFODION-3126
> URL: https://issues.apache.org/jira/browse/TRAFODION-3126
> Project: Apache Trafodion
> Issue Type: Sub-task
> Components: sql-exe
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Priority: Major
>
> New implementation returns the following error when Alluxio file system is
> involved.
> *** ERROR[8447] An error occurred during hdfs access. Error Detail:
> SETUP_HDFS_SCAN java.util.concurrent.ExecutionException:
> java.lang.UnsupportedOperationException: Byte-buffer read unsupported by
> input stream
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> org.trafodion.sql.HDFSClient.trafHdfsReadBuffer(HDFSClient.java:278)
> org.trafodion.sql.HdfsScan.trafHdfsRead(HdfsScan.java:201) Caused by
> java.lang.UnsupportedOperationException: Byte-buffer read unsupported by
> input stream
> org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:150)
> org.trafodion.sql.HDFSClient$HDFSRead.call(HDFSClient.java:172)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
