[
https://issues.apache.org/jira/browse/TRAFODION-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16567789#comment-16567789
]
ASF GitHub Bot commented on TRAFODION-3171:
-------------------------------------------
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1674#discussion_r207440918
--- Diff: core/sql/src/main/java/org/trafodion/sql/HDFSClient.java ---
@@ -240,7 +256,48 @@ int compressedFileRead(int readLenRemain) throws
IOException
return totalReadLen;
}
- native int copyToByteBuffer(ByteBuffer buf, int bufOffset, byte[]
bufArray, int copyLen);
+ /* Trafodion adds record delimiter '\n' while copying it
+ to buffer backing up the ByteBuffer */
+
+ int sequenceFileRead(int readLenRemain) throws IOException
+ {
+ boolean eof = false;
+ byte[] byteArray;
+ int readLen;
+ int totalReadLen = 0;
+ long tempPos;
+ int lenRemain = readLenRemain;
+
+ while (!eof && lenRemain > 0) {
+ tempPos = reader_.getPosition();
+ try {
+ eof = reader_.next(key_, value_);
+ }
+ catch (java.io.EOFException e)
+ {
+ eof = true;
+ break;
+ }
+ byteArray = ((Text)value_).getBytes();
+ readLen = ((Text)value_).getLength();
+ if (readLen <= lenRemain) {
+
+ buf_.put(byteArray, 0, readLen);
+ buf_.put(recDelimiter_);
--- End diff --
Should we be worried that the 1 byte delimiter will put us past us past end
of buffer as suggest in the comment in line 290? (when readLen == lenRemain)
> Refactor Sequence File Reading to use the new implementation
> ------------------------------------------------------------
>
> Key: TRAFODION-3171
> URL: https://issues.apache.org/jira/browse/TRAFODION-3171
> Project: Apache Trafodion
> Issue Type: Sub-task
> Components: sql-exe
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Priority: Major
>
> SequenceFileReader is done via JNI/Java earlier. However, the HdfsScan was
> traversing the older states in ExHdfsScanTcb work method. This new
> implementation should use the current ByteBuffer infrastructure to enable
> obsoleting the older implemenation
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
