[
https://issues.apache.org/jira/browse/TRAFODION-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586116#comment-16586116
]
ASF GitHub Bot commented on TRAFODION-3110:
-------------------------------------------
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1696#discussion_r211312225
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -2455,16 +2455,59 @@ Ex_Lob_Error ExLob::readDataToHdfsFile(char
*tgtFileName, Int64 offset, Int64 s
Int64 srcLen = size;
Int64 srcOffset = offset;
Int64 tgtOffset = 0;
- char *lobData = 0;
+ char *lobData = NULL;
Int64 chunkSize = 0;
hdfsFile fdTgtFile;
char logBuf[4096];
+ writeOperLen = 0;
+ HdfsClient *tgtHdfsClient;
+ HDFS_Client_RetCode hdfsClientRetcode;
+ NABoolean overwrite = TRUE;
+ NABoolean append = FALSE;
+ Int64 remainLen = size;
+ Int64 pos = offset;
+ Int64 readLen;
lobDebugInfo("In ExLob::readDataToHdfsFile",0,__LINE__,lobTrace_);
// open and write to the target file
int openFlags = O_WRONLY;
+ if (! useLibHdfs_) {
+ if (((LobTgtFileFlags)fileflags == Lob_Error_Or_Create) ||
+ ((LobTgtFileFlags)fileflags == Lob_Truncate_Or_Error))
+ overwrite = FALSE;
+ if ((LobTgtFileFlags)fileflags == Lob_Append_Or_Error)
+ append = TRUE;
+ tgtHdfsClient = HdfsClient::newInstance(getLobGlobalHeap(), NULL,
hdfsClientRetcode);
+ ex_assert(hdfsClientRetcode == HDFS_CLIENT_OK, "Internal error:
HdfsClient::newInstance returned an error");
+ if (tgtHdfsClient->hdfsCreate(tgtFileName, overwrite, append, FALSE)
!= HDFS_CLIENT_OK)
+ return LOB_TARGET_FILE_OPEN_ERROR;
+ Int32 bytesRead;
+ Int32 bytesWritten;
+ while (remainLen > 0)
+ {
+ if (remainLen > lobMaxChunkMemLen)
+ readLen = lobMaxChunkMemLen;
+ else
+ readLen = remainLen;
+ if (lobData == NULL)
+ lobData = new (lobGlobalHeap_) char[readLen];
+ bytesRead = hdfsClient_->hdfsRead(pos, lobData, readLen,
hdfsClientRetcode);
+ if (hdfsClientRetcode == HDFS_CLIENT_OK)
--- End diff --
Needs some error handling if retcode is HDFS_CLIENT_OK ?
> Refactor LOB access to use the new implementation of HdfsClient
> ---------------------------------------------------------------
>
> Key: TRAFODION-3110
> URL: https://issues.apache.org/jira/browse/TRAFODION-3110
> Project: Apache Trafodion
> Issue Type: Sub-task
> Components: sql-exe
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Priority: Major
>
> By making LOB to use the new implementation of HdfsClient, the use of libHdfs
> is avoided in yet another feature. This also simplifies the hdfs access
> related code in this feature.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
